* [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
@ 2021-01-05 6:14 Liu, WeiX C
2021-01-05 6:14 ` [PATCH 2/2] MdeModulePkg Lzma: Update LZMA SDK version to 19.00 Liu, WeiX C
2021-01-08 13:54 ` 回复: [edk2-devel] [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version gaoliming
0 siblings, 2 replies; 7+ messages in thread
From: Liu, WeiX C @ 2021-01-05 6:14 UTC (permalink / raw)
To: devel; +Cc: Wei Liu, Liming Gao
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
New formal release in https://www.7-zip.org/sdk.html is 19.00.
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Wei Liu <weix.c.liu@intel.com>
---
BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4 ++--
BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8 ++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5 +++--
BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8 ++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9 +++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 232 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14 +++++++-------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32 ++++++++++++++++----------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 907 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
12 files changed, 877 insertions(+), 575 deletions(-)
diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
index 7cf967a774..b2338e2598 100644
--- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
+++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
@@ -1,3 +1,3 @@
-LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
-was placed in the public domain on 2018-04-30. It was
+LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
+was placed in the public domain on 2019-02-21. It was
released on the http://www.7-zip.org/sdk.html website.
diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
index bebdb9aa84..bf4706ee23 100644
--- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
+++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
@@ -1,9 +1,9 @@
/** @file
LZMA Compress/Decompress tool (LzmaCompress)
- Based on LZMA SDK 18.05:
+ Based on LZMA SDK 19.00:
LzmaUtil.c -- Test application for LZMA compression
- 2018-04-30 : Igor Pavlov : Public domain
+ 2019-02-21 : Igor Pavlov : Public domain
Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
SPDX-License-Identifier: BSD-2-Clause-Patent
@@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate memory";
const char *kDataErrorMessage = "Data error";
const char *kInvalidParamValMessage = "Invalid parameter value";
-static Bool mQuietMode = False;
+static BoolInt mQuietMode = False;
static CONVERTER_TYPE mConType = NoConverter;
UINT64 mDictionarySize = 28;
@@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
CFileOutStream outStream;
int res;
int encodeMode = 0;
- Bool modeWasSet = False;
+ BoolInt modeWasSet = False;
const char *inputFile = NULL;
const char *outputFile = "file.tmp";
int param;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
index 4977cdaa66..593f5aa259 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
@@ -1,5 +1,5 @@
/* 7zTypes.h -- Basic types
-2017-07-17 : Igor Pavlov : Public domain */
+2018-08-04 : Igor Pavlov : Public domain */
#ifndef __7Z_TYPES_H
#define __7Z_TYPES_H
@@ -103,7 +103,8 @@ typedef UInt32 SizeT;
typedef size_t SizeT;
#endif
-typedef int Bool;
+typedef int BoolInt;
+/* typedef BoolInt Bool; */
#define True 1
#define False 0
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
index ed3aa94270..0074c64be9 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
@@ -1,7 +1,7 @@
-#define MY_VER_MAJOR 18
-#define MY_VER_MINOR 05
+#define MY_VER_MAJOR 19
+#define MY_VER_MINOR 00
#define MY_VER_BUILD 0
-#define MY_VERSION_NUMBERS "18.05"
+#define MY_VERSION_NUMBERS "19.00"
#define MY_VERSION MY_VERSION_NUMBERS
#ifdef MY_CPU_NAME
@@ -10,7 +10,7 @@
#define MY_VERSION_CPU MY_VERSION
#endif
-#define MY_DATE "2018-04-30"
+#define MY_DATE "2019-02-21"
#undef MY_COPYRIGHT
#undef MY_VERSION_COPYRIGHT_DATE
#define MY_AUTHOR_NAME "Igor Pavlov"
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
index 7fb27282c7..5f74c1c0cb 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
@@ -1,5 +1,5 @@
/* CpuArch.h -- CPU specific code
-2017-09-04 : Igor Pavlov : Public domain */
+2018-02-18 : Igor Pavlov : Public domain */
#ifndef __CPU_ARCH_H
#define __CPU_ARCH_H
@@ -318,15 +318,16 @@ enum
void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d);
-Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
+BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
int x86cpuid_GetFirm(const Cx86cpuid *p);
#define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) & 0xF))
#define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) & 0xF))
#define x86cpuid_GetStepping(ver) (ver & 0xF)
-Bool CPU_Is_InOrder();
-Bool CPU_Is_Aes_Supported();
+BoolInt CPU_Is_InOrder();
+BoolInt CPU_Is_Aes_Supported();
+BoolInt CPU_IsSupported_PageGB();
#endif
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
index 6ea82a9b53..f89c9ddd7d 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
@@ -1,5 +1,5 @@
/* LzFind.c -- Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-07-08 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -73,7 +73,7 @@ static void MatchFinder_ReadBlock(CMatchFinder *p)
p->streamEndWasReached = 1;
return;
}
-
+
for (;;)
{
Byte *dest = p->buffer + (p->streamPos - p->pos);
@@ -138,7 +138,7 @@ static void MatchFinder_SetDefaultSettings(CMatchFinder *p)
void MatchFinder_Construct(CMatchFinder *p)
{
- UInt32 i;
+ unsigned i;
p->bufferBase = NULL;
p->directInput = 0;
p->hash = NULL;
@@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
for (i = 0; i < 256; i++)
{
- UInt32 r = i;
+ UInt32 r = (UInt32)i;
unsigned j;
for (j = 0; j < 8; j++)
r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
@@ -180,24 +180,24 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
ISzAllocPtr alloc)
{
UInt32 sizeReserv;
-
+
if (historySize > kMaxHistorySize)
{
MatchFinder_Free(p, alloc);
return 0;
}
-
+
sizeReserv = historySize >> 1;
if (historySize >= ((UInt32)3 << 30)) sizeReserv = historySize >> 3;
else if (historySize >= ((UInt32)2 << 30)) sizeReserv = historySize >> 2;
-
+
sizeReserv += (keepAddBufferBefore + matchMaxLen + keepAddBufferAfter) / 2 + (1 << 19);
p->keepSizeBefore = historySize + keepAddBufferBefore + 1;
p->keepSizeAfter = matchMaxLen + keepAddBufferAfter;
-
+
/* we need one additional byte, since we use MoveBlock after pos++ and before dictionary using */
-
+
if (LzInWindow_Create(p, sizeReserv, alloc))
{
UInt32 newCyclicBufferSize = historySize + 1;
@@ -243,7 +243,7 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
p->historySize = historySize;
p->hashSizeSum = hs;
p->cyclicBufferSize = newCyclicBufferSize;
-
+
numSons = newCyclicBufferSize;
if (p->btMode)
numSons <<= 1;
@@ -251,11 +251,11 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
if (p->hash && p->numRefs == newSize)
return 1;
-
+
MatchFinder_FreeThisClassMemory(p, alloc);
p->numRefs = newSize;
p->hash = AllocRefs(newSize, alloc);
-
+
if (p->hash)
{
p->son = p->hash + p->hashSizeSum;
@@ -272,11 +272,11 @@ static void MatchFinder_SetLimits(CMatchFinder *p)
{
UInt32 limit = kMaxValForNormalize - p->pos;
UInt32 limit2 = p->cyclicBufferSize - p->cyclicBufferPos;
-
+
if (limit2 < limit)
limit = limit2;
limit2 = p->streamPos - p->pos;
-
+
if (limit2 <= p->keepSizeAfter)
{
if (limit2 > 0)
@@ -284,10 +284,10 @@ static void MatchFinder_SetLimits(CMatchFinder *p)
}
else
limit2 -= p->keepSizeAfter;
-
+
if (limit2 < limit)
limit = limit2;
-
+
{
UInt32 lenLimit = p->streamPos - p->pos;
if (lenLimit > p->matchMaxLen)
@@ -326,10 +326,10 @@ void MatchFinder_Init_3(CMatchFinder *p, int readData)
p->streamPos = p->cyclicBufferSize;
p->result = SZ_OK;
p->streamEndWasReached = 0;
-
+
if (readData)
MatchFinder_ReadBlock(p);
-
+
MatchFinder_SetLimits(p);
}
@@ -341,7 +341,7 @@ void MatchFinder_Init(CMatchFinder *p)
MatchFinder_Init_3(p, True);
}
-
+
static UInt32 MatchFinder_GetSubValue(CMatchFinder *p)
{
return (p->pos - p->historySize - 1) & kNormalizeMask;
@@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
MatchFinder_ReduceOffsets(p, subValue);
}
+
+MY_NO_INLINE
static void MatchFinder_CheckLimits(CMatchFinder *p)
{
if (p->pos == kMaxValForNormalize)
@@ -379,10 +381,16 @@ static void MatchFinder_CheckLimits(CMatchFinder *p)
MatchFinder_SetLimits(p);
}
-static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
+
+/*
+ (lenLimit > maxLen)
+*/
+MY_FORCE_INLINE
+static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
- UInt32 *distances, UInt32 maxLen)
+ UInt32 *distances, unsigned maxLen)
{
+ /*
son[_cyclicBufferPos] = curMatch;
for (;;)
{
@@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = len;
*distances++ = delta - 1;
if (len == lenLimit)
return distances;
@@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
}
}
}
+ */
+
+ const Byte *lim = cur + lenLimit;
+ son[_cyclicBufferPos] = curMatch;
+ do
+ {
+ UInt32 delta = pos - curMatch;
+ if (delta >= _cyclicBufferSize)
+ break;
+ {
+ ptrdiff_t diff;
+ curMatch = son[_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)];
+ diff = (ptrdiff_t)0 - delta;
+ if (cur[maxLen] == cur[maxLen + diff])
+ {
+ const Byte *c = cur;
+ while (*c == c[diff])
+ {
+ if (++c == lim)
+ {
+ distances[0] = (UInt32)(lim - cur);
+ distances[1] = delta - 1;
+ return distances + 2;
+ }
+ }
+ {
+ unsigned len = (unsigned)(c - cur);
+ if (maxLen < len)
+ {
+ maxLen = len;
+ distances[0] = (UInt32)len;
+ distances[1] = delta - 1;
+ distances += 2;
+ }
+ }
+ }
+ }
+ }
+ while (--cutValue);
+
+ return distances;
}
+
+MY_FORCE_INLINE
UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
UInt32 *distances, UInt32 maxLen)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
return distances;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = pair[0];
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = (UInt32)len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
+ *ptr1 = pair0;
*ptr0 = pair[1];
return distances;
}
@@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
return;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
if (pb[len] == cur[len])
{
while (++len != lenLimit)
@@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
p->buffer++; \
if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
-#define MOVE_POS_RET MOVE_POS return offset;
+#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define GET_MATCHES_HEADER2(minLen, ret_op) \
- UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
- lenLimit = p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
+ unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
+ lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
cur = p->buffer;
#define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen, return 0)
@@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos, p->cyclicBufferSize, p->cutValue
#define GET_MATCHES_FOOTER(offset, maxLen) \
- offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
- distances + offset, maxLen) - distances); MOVE_POS_RET;
+ offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch, MF_PARAMS(p), \
+ distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
#define SKIP_FOOTER \
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
#define UPDATE_maxLen { \
ptrdiff_t diff = (ptrdiff_t)0 - d2; \
const Byte *c = cur + maxLen; \
const Byte *lim = cur + lenLimit; \
for (; c != lim; c++) if (*(c + diff) != *c) break; \
- maxLen = (UInt32)(c - cur); }
+ maxLen = (unsigned)(c - cur); }
static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(2)
HASH2_CALC;
curMatch = p->hash[hv];
@@ -561,7 +615,7 @@ static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
@@ -572,7 +626,8 @@ UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, d2, maxLen, offset, pos;
+ UInt32 h2, d2, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(3)
@@ -584,7 +639,7 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
d2 = pos - hash[h2];
curMatch = (hash + kFix3HashSize)[hv];
-
+
hash[h2] = pos;
(hash + kFix3HashSize)[hv] = pos;
@@ -594,22 +649,23 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
UPDATE_maxLen
- distances[0] = maxLen;
+ distances[0] = (UInt32)maxLen;
distances[1] = d2 - 1;
offset = 2;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
-
+
GET_MATCHES_FOOTER(offset, maxLen)
}
static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -618,25 +674,26 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[hv] = pos;
maxLen = 0;
offset = 0;
-
+
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
-
+
if (d2 != d3 && d3 < p->cyclicBufferSize && *(cur - d3) == *cur)
{
maxLen = 3;
@@ -644,21 +701,21 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d3;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
-
+
if (maxLen < 3)
maxLen = 3;
-
+
GET_MATCHES_FOOTER(offset, maxLen)
}
@@ -674,13 +731,13 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
d4 = pos - (hash + kFix4HashSize)[h4];
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[h4] = pos;
(hash + kFix5HashSize)[hv] = pos;
@@ -710,7 +767,7 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset = 2;
d2 = d3;
}
-
+
if (d2 != d4 && d4 < p->cyclicBufferSize
&& *(cur - d4) == *cur
&& *(cur - d4 + 3) == *(cur + 3))
@@ -720,7 +777,7 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d4;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
@@ -734,14 +791,15 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (maxLen < 4)
maxLen = 4;
-
+
GET_MATCHES_FOOTER(offset, maxLen)
}
*/
static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -749,13 +807,12 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
-
- d2 = pos - hash[ h2];
+
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
-
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[hv] = pos;
@@ -764,11 +821,12 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
-
+
if (d2 != d3 && d3 < p->cyclicBufferSize && *(cur - d3) == *cur)
{
maxLen = 3;
@@ -776,22 +834,22 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d3;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
p->son[p->cyclicBufferPos] = curMatch;
MOVE_POS_RET;
}
}
-
+
if (maxLen < 3)
maxLen = 3;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances + offset, maxLen) - (distances));
MOVE_POS_RET
}
@@ -807,14 +865,14 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
-
- d2 = pos - hash[ h2];
+
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
d4 = pos - (hash + kFix4HashSize)[h4];
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[h4] = pos;
(hash + kFix5HashSize)[hv] = pos;
@@ -844,7 +902,7 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset = 2;
d2 = d3;
}
-
+
if (d2 != d4 && d4 < p->cyclicBufferSize
&& *(cur - d4) == *cur
&& *(cur - d4 + 3) == *(cur + 3))
@@ -854,7 +912,7 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d4;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
@@ -865,7 +923,7 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
MOVE_POS_RET;
}
}
-
+
if (maxLen < 4)
maxLen = 4;
@@ -877,12 +935,12 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
p->hash[hv] = p->pos;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances, 2) - (distances));
MOVE_POS_RET
}
@@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH4_CALC;
hash = p->hash;
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[hv] = p->pos;
SKIP_FOOTER
@@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH5_CALC;
hash = p->hash;
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[h4] =
(hash + kFix5HashSize)[hv] = p->pos;
@@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH4_CALC;
hash = p->hash;
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[hv] = p->pos;
p->son[p->cyclicBufferPos] = curMatch;
@@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH5_CALC;
hash = p->hash;
curMatch = hash + kFix5HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[h4] =
(hash + kFix5HashSize)[hv] = p->pos;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
index 2563824fcd..df32146f92 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
@@ -1,5 +1,5 @@
/* LzFindMt.c -- multithreaded Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-12-29 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -232,38 +232,57 @@ static void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
#define kEmptyHashValue 0
-/* #define MFMT_GM_INLINE */
+#define MFMT_GM_INLINE
#ifdef MFMT_GM_INLINE
-#define NO_INLINE MY_FAST_CALL
+/*
+ we use size_t for _cyclicBufferPos instead of UInt32
+ to eliminate "movsx" BUG in old MSVC x64 compiler.
+*/
-static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
- UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
- UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit, UInt32 size, UInt32 *posRes)
+MY_NO_INLINE
+static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
+ size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
+ UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32 *limit, UInt32 size, UInt32 *posRes)
{
do
{
- UInt32 *distances = _distances + 1;
- UInt32 curMatch = pos - *hash++;
+ UInt32 *_distances = ++distances;
+ UInt32 delta = *hash++;
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
UInt32 cutValue = _cutValue;
- UInt32 maxLen = _maxLen;
- for (;;)
+ unsigned maxLen = (unsigned)_maxLen;
+
+ /*
+ if (size > 1)
{
- UInt32 delta = pos - curMatch;
- if (cutValue-- == 0 || delta >= _cyclicBufferSize)
+ UInt32 delta = *hash;
+ if (delta < _cyclicBufferSize)
{
- *ptr0 = *ptr1 = kEmptyHashValue;
- break;
+ UInt32 cyc1 = _cyclicBufferPos + 1;
+ CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ? _cyclicBufferSize : 0)) << 1);
+ Byte b = *(cur + 1 - delta);
+ _distances[0] = pair[0];
+ _distances[1] = b;
}
+ }
+ */
+ if (cutValue == 0 || delta >= _cyclicBufferSize)
+ {
+ *ptr0 = *ptr1 = kEmptyHashValue;
+ }
+ else
+ for(;;)
+ {
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = *pair;
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
- *ptr0 = pair[1];
+ UInt32 pair1 = pair[1];
+ *ptr1 = pair0;
+ *ptr0 = pair1;
break;
}
}
}
- if (pb[len] < cur[len])
- {
- *ptr1 = curMatch;
- ptr1 = pair + 1;
- curMatch = *ptr1;
- len1 = len;
- }
- else
{
- *ptr0 = curMatch;
- ptr0 = pair;
- curMatch = *ptr0;
- len0 = len;
+ UInt32 curMatch = pos - delta;
+ // delta = pos - *pair;
+ // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >> 31];
+ if (pb[len] < cur[len])
+ {
+ delta = pos - pair[1];
+ *ptr1 = curMatch;
+ ptr1 = pair + 1;
+ len1 = len;
+ }
+ else
+ {
+ delta = pos - *pair;
+ *ptr0 = curMatch;
+ ptr0 = pair;
+ len0 = len;
+ }
}
}
+ if (--cutValue == 0 || delta >= _cyclicBufferSize)
+ {
+ *ptr0 = *ptr1 = kEmptyHashValue;
+ break;
+ }
}
pos++;
_cyclicBufferPos++;
cur++;
{
UInt32 num = (UInt32)(distances - _distances);
- *_distances = num - 1;
- _distances += num;
- limit -= num;
+ _distances[-1] = num;
}
}
- while (limit > 0 && --size != 0);
+ while (distances < limit && --size != 0);
*posRes = pos;
- return limit;
+ return distances;
}
#endif
+
+
static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
{
UInt32 numProcessed = 0;
UInt32 curPos = 2;
- UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
+ UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
distances[1] = p->hashNumAvail;
@@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
#else
{
UInt32 posRes;
- curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
- distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
+ curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
+ distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos,
+ distances + limit,
+ size, &posRes) - distances);
p->hashBufPos += posRes - pos;
cyclicBufferPos += posRes - pos;
p->buffer += posRes - pos;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
index 3d86c788f3..fdd17008c2 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
@@ -1,5 +1,5 @@
/* LzFindMt.h -- multithreaded Match finder for LZ algorithms
-2017-04-03 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#ifndef __LZ_FIND_MT_H
#define __LZ_FIND_MT_H
@@ -19,10 +19,10 @@ EXTERN_C_BEGIN
typedef struct _CMtSync
{
- Bool wasCreated;
- Bool needStart;
- Bool exit;
- Bool stopWriting;
+ BoolInt wasCreated;
+ BoolInt needStart;
+ BoolInt exit;
+ BoolInt stopWriting;
CThread thread;
CAutoResetEvent canStart;
@@ -30,8 +30,8 @@ typedef struct _CMtSync
CAutoResetEvent wasStopped;
CSemaphore freeSemaphore;
CSemaphore filledSemaphore;
- Bool csWasInitialized;
- Bool csWasEntered;
+ BoolInt csWasInitialized;
+ BoolInt csWasEntered;
CCriticalSection cs;
UInt32 numProcessedBlocks;
} CMtSync;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
index 962b94bb63..4d1576419f 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
@@ -1,13 +1,13 @@
/* LzmaDec.c -- LZMA Decoder
-2018-02-28 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
+#include <string.h>
+
/* #include "CpuArch.h" */
#include "LzmaDec.h"
-#include <string.h>
-
#define kNumTopBits 24
#define kTopValue ((UInt32)1 << kNumTopBits)
@@ -19,7 +19,7 @@
#define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
#define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt - (ttt >> kNumMoveBits));
#define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
@@ -66,7 +66,7 @@
#define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit) return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0_CHECK range = bound;
#define UPDATE_1_CHECK range -= bound; code -= bound;
#define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
@@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
curLen = ((rem < len) ? (unsigned)rem : len);
pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
- processedPos += curLen;
+ processedPos += (UInt32)curLen;
len -= curLen;
if (curLen <= dicBufSize - pos)
@@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
Byte *dest = dic + dicPos;
ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
const Byte *lim = dest + curLen;
- dicPos += curLen;
+ dicPos += (SizeT)curLen;
do
*(dest) = (Byte)*(dest + src);
while (++dest != lim);
@@ -572,14 +572,14 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
p->buf = buf;
p->range = range;
p->code = code;
- p->remainLen = len;
+ p->remainLen = (UInt32)len;
p->dicPos = dicPos;
p->processedPos = processedPos;
p->reps[0] = rep0;
p->reps[1] = rep1;
p->reps[2] = rep2;
p->reps[3] = rep3;
- p->state = state;
+ p->state = (UInt32)state;
return SZ_OK;
}
@@ -601,8 +601,8 @@ static void MY_FAST_CALL LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
p->checkDicSize = p->prop.dicSize;
- p->processedPos += len;
- p->remainLen -= len;
+ p->processedPos += (UInt32)len;
+ p->remainLen -= (UInt32)len;
while (len != 0)
{
len--;
@@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, const Byte *buf, SizeT inS
}
-void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
+void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt initState)
{
p->remainLen = kMatchSpecLenStart + 1;
p->tempBufSize = 0;
@@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
p->tempBufSize = rem;
if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
{
- int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
+ int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, (SizeT)rem);
if (dummyRes == DUMMY_ERROR)
{
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
*status = LZMA_STATUS_NEEDS_MORE_INPUT;
return SZ_OK;
}
@@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
return SZ_ERROR_FAIL; /* some internal error */
lookAhead -= rem;
}
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
src += lookAhead;
- inSize -= lookAhead;
+ inSize -= (SizeT)lookAhead;
p->tempBufSize = 0;
}
}
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
index e281716fee..4e9b499f8d 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
@@ -1,5 +1,5 @@
/* LzmaEnc.c -- LZMA Encoder
-2018-04-29 : Igor Pavlov : Public domain */
+2019-01-10: Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
#define kNumBitPriceShiftBits 4
#define kBitPrice (1 << kNumBitPriceShiftBits)
+#define REP_LEN_COUNT 64
+
void LzmaEncProps_Init(CLzmaEncProps *p)
{
p->level = 5;
@@ -52,7 +54,7 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p)
int level = p->level;
if (level < 0) level = 5;
p->level = level;
-
+
if (p->dictSize == 0) p->dictSize = (level <= 5 ? (1 << (level * 2 + 14)) : (level <= 7 ? (1 << 25) : (1 << 26)));
if (p->dictSize > p->reduceSize)
{
@@ -74,7 +76,7 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p)
if (p->btMode < 0) p->btMode = (p->algo == 0 ? 0 : 1);
if (p->numHashBytes < 0) p->numHashBytes = 4;
if (p->mc == 0) p->mc = (16 + (p->fb >> 1)) >> (p->btMode ? 0 : 1);
-
+
if (p->numThreads < 0)
p->numThreads =
#ifndef _7ZIP_ST
@@ -124,7 +126,7 @@ static void LzmaEnc_FastPosInit(Byte *g_FastPos)
g_FastPos[0] = 0;
g_FastPos[1] = 1;
g_FastPos += 2;
-
+
for (slot = 2; slot < kNumLogBits * 2; slot++)
{
size_t k = ((size_t)1 << ((slot >> 1) - 1));
@@ -183,8 +185,11 @@ typedef struct
} COptimal;
-#define kNumOpts (1 << 12)
-#define kPackReserve (1 + kNumOpts * 2)
+// 18.06
+#define kNumOpts (1 << 11)
+#define kPackReserve (kNumOpts * 8)
+// #define kNumOpts (1 << 12)
+// #define kPackReserve (1 + kNumOpts * 2)
#define kNumLenToPosStates 4
#define kNumPosSlotBits 6
@@ -236,10 +241,18 @@ typedef struct
typedef struct
{
unsigned tableSize;
- unsigned counters[LZMA_NUM_PB_STATES_MAX];
UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
+ // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols * 2];
+ // UInt32 prices2[kLenNumSymbolsTotal];
} CLenPriceEnc;
+#define GET_PRICE_LEN(p, posState, len) \
+ ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
+
+/*
+#define GET_PRICE_LEN(p, posState, len) \
+ ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) - 2) & (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >> 9)))
+*/
typedef struct
{
@@ -273,7 +286,7 @@ typedef struct
CLzmaProb posSlotEncoder[kNumLenToPosStates][1 << kNumPosSlotBits];
CLzmaProb posEncoders[kNumFullDistances];
-
+
CLenEnc lenProbs;
CLenEnc repLenProbs;
@@ -308,16 +321,18 @@ typedef struct
unsigned lc, lp, pb;
unsigned lclp;
- Bool fastMode;
- Bool writeEndMark;
- Bool finished;
- Bool multiThread;
- Bool needInit;
+ BoolInt fastMode;
+ BoolInt writeEndMark;
+ BoolInt finished;
+ BoolInt multiThread;
+ BoolInt needInit;
+ // BoolInt _maxMode;
UInt64 nowPos64;
-
+
unsigned matchPriceCount;
- unsigned alignPriceCount;
+ // unsigned alignPriceCount;
+ int repLenEncCounter;
unsigned distTableSize;
@@ -325,7 +340,7 @@ typedef struct
SRes result;
#ifndef _7ZIP_ST
- Bool mtMode;
+ BoolInt mtMode;
// begin of CMatchFinderMt is used in LZ thread
CMatchFinderMt matchFinderMt;
// end of CMatchFinderMt is used in BT and HASH threads
@@ -336,7 +351,7 @@ typedef struct
#ifndef _7ZIP_ST
Byte pad[128];
#endif
-
+
// LZ thread
CProbPrice ProbPrices[kBitModelTotal >> kNumMoveReducingBits];
@@ -355,7 +370,7 @@ typedef struct
CLzmaProb isRep0Long[kNumStates][LZMA_NUM_PB_STATES_MAX];
CLzmaProb posSlotEncoder[kNumLenToPosStates][1 << kNumPosSlotBits];
CLzmaProb posEncoders[kNumFullDistances];
-
+
CLenEnc lenProbs;
CLenEnc repLenProbs;
@@ -383,9 +398,9 @@ void LzmaEnc_SaveState(CLzmaEncHandle pp)
{
CLzmaEnc *p = (CLzmaEnc *)pp;
CSaveState *dest = &p->saveState;
-
+
dest->state = p->state;
-
+
dest->lenProbs = p->lenProbs;
dest->repLenProbs = p->repLenProbs;
@@ -414,9 +429,9 @@ void LzmaEnc_RestoreState(CLzmaEncHandle pp)
dest->lenProbs = p->lenProbs;
dest->repLenProbs = p->repLenProbs;
-
+
COPY_ARR(dest, p, reps);
-
+
COPY_ARR(dest, p, posAlignEncoder);
COPY_ARR(dest, p, isRep);
COPY_ARR(dest, p, isRepG0);
@@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEncProps *props2)
p->lp = props.lp;
p->pb = props.pb;
p->fastMode = (props.algo == 0);
+ // p->_maxMode = True;
p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
{
unsigned numHashBytes = 4;
@@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
#ifdef _LZMA_ENC_USE_BRANCH
-#define RC_BIT(p, prob, symbol) { \
+#define RC_BIT(p, prob, bit) { \
RC_BIT_PRE(p, prob) \
- if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >> kNumMoveBits; } \
+ if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >> kNumMoveBits; } \
else { (p)->low += newBound; range -= newBound; ttt -= ttt >> kNumMoveBits; } \
*(prob) = (CLzmaProb)ttt; \
RC_NORM(p) \
@@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
#else
-#define RC_BIT(p, prob, symbol) { \
+#define RC_BIT(p, prob, bit) { \
UInt32 mask; \
RC_BIT_PRE(p, prob) \
- mask = 0 - (UInt32)symbol; \
+ mask = 0 - (UInt32)bit; \
range &= mask; \
mask &= newBound; \
range -= mask; \
(p)->low += mask; \
- mask = (UInt32)symbol - 1; \
+ mask = (UInt32)bit - 1; \
range += newBound & mask; \
mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
mask += ((1 << kNumMoveBits) - 1); \
@@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p, CLzmaProb *prob)
p->range = range;
}
-static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
+static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
{
UInt32 range = p->range;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
UInt32 ttt, newBound;
- // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
- CLzmaProb *prob = probs + (symbol >> 8);
- UInt32 bit = (symbol >> 7) & 1;
- symbol <<= 1;
+ // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
+ CLzmaProb *prob = probs + (sym >> 8);
+ UInt32 bit = (sym >> 7) & 1;
+ sym <<= 1;
RC_BIT(p, prob, bit);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
p->range = range;
}
-static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol, UInt32 matchByte)
+static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 sym, UInt32 matchByte)
{
UInt32 range = p->range;
UInt32 offs = 0x100;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
UInt32 ttt, newBound;
CLzmaProb *prob;
UInt32 bit;
matchByte <<= 1;
- // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol >> 8)), (symbol >> 7) & 1);
- prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
- bit = (symbol >> 7) & 1;
- symbol <<= 1;
- offs &= ~(matchByte ^ symbol);
+ // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >> 8)), (sym >> 7) & 1);
+ prob = probs + (offs + (matchByte & offs) + (sym >> 8));
+ bit = (sym >> 7) & 1;
+ sym <<= 1;
+ offs &= ~(matchByte ^ sym);
RC_BIT(p, prob, bit);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
p->range = range;
}
@@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice *ProbPrices)
}
-#define GET_PRICE(prob, symbol) \
- p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
+#define GET_PRICE(prob, bit) \
+ p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
-#define GET_PRICEa(prob, symbol) \
- ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
+#define GET_PRICEa(prob, bit) \
+ ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
#define GET_PRICE_0(prob) p->ProbPrices[(prob) >> kNumMoveReducingBits]
#define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
@@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice *ProbPrices)
#define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
-static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol, const CProbPrice *ProbPrices)
+static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const CProbPrice *ProbPrices)
{
UInt32 price = 0;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
- unsigned bit = symbol & 1;
- symbol >>= 1;
- price += GET_PRICEa(probs[symbol], bit);
+ unsigned bit = sym & 1;
+ sym >>= 1;
+ price += GET_PRICEa(probs[sym], bit);
}
- while (symbol >= 2);
+ while (sym >= 2);
return price;
}
-static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32 symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
+static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32 sym, UInt32 matchByte, const CProbPrice *ProbPrices)
{
UInt32 price = 0;
UInt32 offs = 0x100;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
matchByte <<= 1;
- price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)], (symbol >> 7) & 1);
- symbol <<= 1;
- offs &= ~(matchByte ^ symbol);
+ price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)], (sym >> 7) & 1);
+ sym <<= 1;
+ offs &= ~(matchByte ^ sym);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
return price;
}
-static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, unsigned numBits, UInt32 symbol)
+static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, unsigned numBits, unsigned sym)
{
UInt32 range = rc->range;
unsigned m = 1;
do
{
UInt32 ttt, newBound;
- unsigned bit = symbol & 1;
+ unsigned bit = sym & 1;
// RangeEnc_EncodeBit(rc, probs + m, bit);
- symbol >>= 1;
+ sym >>= 1;
RC_BIT(rc, probs + m, bit);
m = (m << 1) | bit;
}
@@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
p->high[i] = kProbInitValue;
}
-static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol, unsigned posState)
+static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym, unsigned posState)
{
UInt32 range, ttt, newBound;
CLzmaProb *probs = p->low;
range = rc->range;
RC_BIT_PRE(rc, probs);
- if (symbol >= kLenNumLowSymbols)
+ if (sym >= kLenNumLowSymbols)
{
RC_BIT_1(rc, probs);
probs += kLenNumLowSymbols;
RC_BIT_PRE(rc, probs);
- if (symbol >= kLenNumLowSymbols * 2)
+ if (sym >= kLenNumLowSymbols * 2)
{
RC_BIT_1(rc, probs);
rc->range = range;
- // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol - kLenNumLowSymbols * 2);
- LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
+ // RcTree_Encode(rc, p->high, kLenNumHighBits, sym - kLenNumLowSymbols * 2);
+ LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
return;
}
- symbol -= kLenNumLowSymbols;
+ sym -= kLenNumLowSymbols;
}
- // RcTree_Encode(rc, probs + (posState << kLenNumLowBits), kLenNumLowBits, symbol);
+ // RcTree_Encode(rc, probs + (posState << kLenNumLowBits), kLenNumLowBits, sym);
{
unsigned m;
unsigned bit;
RC_BIT_0(rc, probs);
probs += (posState << (1 + kLenNumLowBits));
- bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
- bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
- bit = symbol & 1; RC_BIT(rc, probs + m, bit);
+ bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
+ bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
+ bit = sym & 1; RC_BIT(rc, probs + m, bit);
rc->range = range;
}
}
@@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs, UInt32 startPrice, UInt32 *price
}
-MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
- CLenPriceEnc *p, unsigned posState,
+MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
+ CLenPriceEnc *p,
+ unsigned numPosStates,
const CLenEnc *enc,
const CProbPrice *ProbPrices)
{
- // int y; for (y = 0; y < 100; y++) {
- UInt32 a;
- unsigned i, numSymbols;
+ UInt32 b;
+
+ {
+ unsigned prob = enc->low[0];
+ UInt32 a, c;
+ unsigned posState;
+ b = GET_PRICEa_1(prob);
+ a = GET_PRICEa_0(prob);
+ c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
+ for (posState = 0; posState < numPosStates; posState++)
+ {
+ UInt32 *prices = p->prices[posState];
+ const CLzmaProb *probs = enc->low + (posState << (1 + kLenNumLowBits));
+ SetPrices_3(probs, a, prices, ProbPrices);
+ SetPrices_3(probs + kLenNumLowSymbols, c, prices + kLenNumLowSymbols, ProbPrices);
+ }
+ }
- UInt32 *prices = p->prices[posState];
+ /*
{
- const CLzmaProb *probs = enc->low + (posState << (1 + kLenNumLowBits));
- SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
+ unsigned i;
+ UInt32 b;
+ a = GET_PRICEa_0(enc->low[0]);
+ for (i = 0; i < kLenNumLowSymbols; i++)
+ p->prices2[i] = a;
a = GET_PRICEa_1(enc->low[0]);
- SetPrices_3(probs + kLenNumLowSymbols, a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices + kLenNumLowSymbols, ProbPrices);
+ b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
+ for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
+ p->prices2[i] = b;
a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
}
- numSymbols = p->tableSize;
- p->counters[posState] = numSymbols;
- for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
+ */
+
+ // p->counter = numSymbols;
+ // p->counter = 64;
+
{
- prices[i] = a +
- // RcTree_GetPrice(enc->high, kLenNumHighBits, i - kLenNumLowSymbols * 2, ProbPrices);
- LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2, ProbPrices);
- /*
- unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
- UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1, sym, ProbPrices);
- UInt32 prob = enc->high[(1 << 7) + sym];
- prices[i ] = price + GET_PRICEa_0(prob);
- prices[i + 1] = price + GET_PRICEa_1(prob);
- */
- }
- // }
-}
+ unsigned i = p->tableSize;
-static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
- const CLenEnc *enc,
- const CProbPrice *ProbPrices)
-{
- unsigned posState;
- for (posState = 0; posState < numPosStates; posState++)
- LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
-}
+ if (i > kLenNumLowSymbols * 2)
+ {
+ const CLzmaProb *probs = enc->high;
+ UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
+ i -= kLenNumLowSymbols * 2 - 1;
+ i >>= 1;
+ b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
+ do
+ {
+ /*
+ p->prices2[i] = a +
+ // RcTree_GetPrice(enc->high, kLenNumHighBits, i - kLenNumLowSymbols * 2, ProbPrices);
+ LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
+ */
+ // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1, sym, ProbPrices);
+ unsigned sym = --i + (1 << (kLenNumHighBits - 1));
+ UInt32 price = b;
+ do
+ {
+ unsigned bit = sym & 1;
+ sym >>= 1;
+ price += GET_PRICEa(probs[sym], bit);
+ }
+ while (sym >= 2);
+
+ {
+ unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits - 1))];
+ prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
+ prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
+ }
+ }
+ while (i);
+ {
+ unsigned posState;
+ size_t num = (p->tableSize - kLenNumLowSymbols * 2) * sizeof(p->prices[0][0]);
+ for (posState = 1; posState < numPosStates; posState++)
+ memcpy(p->prices[posState] + kLenNumLowSymbols * 2, p->prices[0] + kLenNumLowSymbols * 2, num);
+ }
+ }
+ }
+}
/*
#ifdef SHOW_STAT
@@ -914,21 +973,21 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
printf("\n MovePos %u", num);
#endif
*/
-
+
#define MOVE_POS(p, num) { \
p->additionalOffset += (num); \
- p->matchFinder.Skip(p->matchFinderObj, (num)); }
+ p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
{
unsigned numPairs;
-
+
p->additionalOffset++;
p->numAvail = p->matchFinder.GetNumAvailableBytes(p->matchFinderObj);
numPairs = p->matchFinder.GetMatches(p->matchFinderObj, p->matches);
*numPairsRes = numPairs;
-
+
#ifdef SHOW_STAT
printf("\n i = %u numPairs = %u ", g_STAT_OFFSET, numPairs / 2);
g_STAT_OFFSET++;
@@ -938,7 +997,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
printf("%2u %6u | ", p->matches[i], p->matches[i + 1]);
}
#endif
-
+
if (numPairs == 0)
return 0;
{
@@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
const Byte *p2 = p1 + len;
ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
const Byte *lim = p1 + numAvail;
- for (; p2 != lim && *p2 == p2[dif]; p2++);
+ for (; p2 != lim && *p2 == p2[dif]; p2++)
+ {}
return (unsigned)(p2 - p1);
}
}
@@ -976,8 +1036,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
+ GET_PRICE_1(p->isRep0Long[state][posState])) \
+ GET_PRICE_1(p->isRep[state]) \
+ GET_PRICE_0(p->isRepG0[state])
-
+MY_FORCE_INLINE
static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex, size_t state, size_t posState)
{
UInt32 price;
@@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p, unsigned cur)
for (;;)
{
UInt32 dist = p->opt[cur].dist;
- UInt32 len = p->opt[cur].len;
- UInt32 extra = p->opt[cur].extra;
+ unsigned len = (unsigned)p->opt[cur].len;
+ unsigned extra = (unsigned)p->opt[cur].extra;
cur -= len;
if (extra)
{
wr--;
- p->opt[wr].len = len;
+ p->opt[wr].len = (UInt32)len;
cur -= extra;
len = extra;
if (extra == 1)
@@ -1042,10 +1102,10 @@ static unsigned Backward(CLzmaEnc *p, unsigned cur)
p->optCur = wr;
return len;
}
-
+
wr--;
p->opt[wr].dist = dist;
- p->opt[wr].len = len;
+ p->opt[wr].len = (UInt32)len;
}
}
@@ -1068,9 +1128,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
UInt32 matchPrice, repMatchPrice;
const Byte *data;
Byte curByte, matchByte;
-
+
p->optCur = p->optEnd = 0;
-
+
if (p->additionalOffset == 0)
mainLen = ReadMatchDistances(p, &numPairs);
else
@@ -1078,7 +1138,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
mainLen = p->longestMatchLen;
numPairs = p->numPairs;
}
-
+
numAvail = p->numAvail;
if (numAvail < 2)
{
@@ -1087,10 +1147,10 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
if (numAvail > LZMA_MATCH_LEN_MAX)
numAvail = LZMA_MATCH_LEN_MAX;
-
+
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
repMaxIndex = 0;
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned len;
@@ -1102,43 +1162,48 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
repLens[i] = 0;
continue;
}
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
repLens[i] = len;
if (len > repLens[repMaxIndex])
repMaxIndex = i;
}
-
+
if (repLens[repMaxIndex] >= p->numFastBytes)
{
unsigned len;
- p->backRes = repMaxIndex;
+ p->backRes = (UInt32)repMaxIndex;
len = repLens[repMaxIndex];
MOVE_POS(p, len - 1)
return len;
}
-
+
matches = p->matches;
-
+
if (mainLen >= p->numFastBytes)
{
p->backRes = matches[(size_t)numPairs - 1] + LZMA_NUM_REPS;
MOVE_POS(p, mainLen - 1)
return mainLen;
}
-
+
curByte = *data;
matchByte = *(data - reps[0]);
-
- if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] < 2)
+
+ last = repLens[repMaxIndex];
+ if (last <= mainLen)
+ last = mainLen;
+
+ if (last < 2 && curByte != matchByte)
{
p->backRes = MARK_LIT;
return 1;
}
-
+
p->opt[0].state = (CState)p->state;
-
+
posState = (position & p->pbMask);
-
+
{
const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
p->opt[1].price = GET_PRICE_0(p->isMatch[p->state][posState]) +
@@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
LitEnc_Matched_GetPrice(probs, curByte, matchByte, p->ProbPrices) :
LitEnc_GetPrice(probs, curByte, p->ProbPrices));
}
-
+
MakeAs_Lit(&p->opt[1]);
-
+
matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
-
- if (matchByte == curByte)
+
+ // 18.06
+ if (matchByte == curByte && repLens[0] == 0)
{
UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, p->state, posState);
if (shortRepPrice < p->opt[1].price)
@@ -1160,32 +1226,22 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
p->opt[1].price = shortRepPrice;
MakeAs_ShortRep(&p->opt[1]);
}
+ if (last < 2)
+ {
+ p->backRes = p->opt[1].dist;
+ return 1;
+ }
}
-
- last = (mainLen >= repLens[repMaxIndex] ? mainLen : repLens[repMaxIndex]);
-
- if (last < 2)
- {
- p->backRes = p->opt[1].dist;
- return 1;
- }
-
+
p->opt[1].len = 1;
-
+
p->opt[0].reps[0] = reps[0];
p->opt[0].reps[1] = reps[1];
p->opt[0].reps[2] = reps[2];
p->opt[0].reps[3] = reps[3];
-
- {
- unsigned len = last;
- do
- p->opt[len--].price = kInfinityPrice;
- while (len >= 2);
- }
// ---------- REP ----------
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned repLen = repLens[i];
@@ -1195,58 +1251,61 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
do
{
- UInt32 price2 = price + p->repLenEnc.prices[posState][(size_t)repLen - 2];
+ UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState, repLen);
COptimal *opt = &p->opt[repLen];
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = repLen;
- opt->dist = i;
+ opt->len = (UInt32)repLen;
+ opt->dist = (UInt32)i;
opt->extra = 0;
}
}
while (--repLen >= 2);
}
-
-
+
+
// ---------- MATCH ----------
{
- unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
+ unsigned len = repLens[0] + 1;
if (len <= mainLen)
{
unsigned offs = 0;
UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[p->state]);
- while (len > matches[offs])
- offs += 2;
-
+ if (len < 2)
+ len = 2;
+ else
+ while (len > matches[offs])
+ offs += 2;
+
for (; ; len++)
{
COptimal *opt;
UInt32 dist = matches[(size_t)offs + 1];
- UInt32 price2 = normalMatchPrice + p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
+ UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc, posState, len);
unsigned lenToPosState = GetLenToPosState(len);
-
+
if (dist < kNumFullDistances)
- price2 += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
+ price += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
else
{
unsigned slot;
GetPosSlot2(dist, slot);
- price2 += p->alignPrices[dist & kAlignMask];
- price2 += p->posSlotPrices[lenToPosState][slot];
+ price += p->alignPrices[dist & kAlignMask];
+ price += p->posSlotPrices[lenToPosState][slot];
}
-
+
opt = &p->opt[len];
-
- if (price2 < opt->price)
+
+ if (price < opt->price)
{
- opt->price = price2;
- opt->len = len;
+ opt->price = price;
+ opt->len = (UInt32)len;
opt->dist = dist + LZMA_NUM_REPS;
opt->extra = 0;
}
-
+
if (len == matches[offs])
{
offs += 2;
@@ -1256,7 +1315,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
-
+
cur = 0;
@@ -1272,37 +1331,73 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
-
+
// ---------- Optimal Parsing ----------
for (;;)
{
- UInt32 numAvail, numAvailFull;
+ unsigned numAvail;
+ UInt32 numAvailFull;
unsigned newLen, numPairs, prev, state, posState, startLen;
- UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
- Bool nextIsLit;
+ UInt32 litPrice, matchPrice, repMatchPrice;
+ BoolInt nextIsLit;
Byte curByte, matchByte;
const Byte *data;
COptimal *curOpt, *nextOpt;
if (++cur == last)
- return Backward(p, cur);
+ break;
+
+ // 18.06
+ if (cur >= kNumOpts - 64)
+ {
+ unsigned j, best;
+ UInt32 price = p->opt[cur].price;
+ best = cur;
+ for (j = cur + 1; j <= last; j++)
+ {
+ UInt32 price2 = p->opt[j].price;
+ if (price >= price2)
+ {
+ price = price2;
+ best = j;
+ }
+ }
+ {
+ unsigned delta = best - cur;
+ if (delta != 0)
+ {
+ MOVE_POS(p, delta);
+ }
+ }
+ cur = best;
+ break;
+ }
newLen = ReadMatchDistances(p, &numPairs);
-
+
if (newLen >= p->numFastBytes)
{
p->numPairs = numPairs;
p->longestMatchLen = newLen;
- return Backward(p, cur);
+ break;
}
-
+
curOpt = &p->opt[cur];
+
+ position++;
+
+ // we need that check here, if skip_items in p->opt are possible
+ /*
+ if (curOpt->price >= kInfinityPrice)
+ continue;
+ */
+
prev = cur - curOpt->len;
-
+
if (curOpt->len == 1)
{
- state = p->opt[prev].state;
+ state = (unsigned)p->opt[prev].state;
if (IsShortRep(curOpt))
state = kShortRepNextStates[state];
else
@@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (curOpt->extra)
{
- prev -= curOpt->extra;
+ prev -= (unsigned)curOpt->extra;
state = kState_RepAfterLit;
if (curOpt->extra == 1)
- state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit : kState_MatchAfterLit;
+ state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit : kState_MatchAfterLit);
}
else
{
- state = p->opt[prev].state;
+ state = (unsigned)p->opt[prev].state;
if (dist < LZMA_NUM_REPS)
state = kRepNextStates[state];
else
@@ -1368,7 +1463,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
reps[3] = prevOpt->reps[2];
}
}
-
+
curOpt->state = (CState)state;
curOpt->reps[0] = reps[0];
curOpt->reps[1] = reps[1];
@@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
curByte = *data;
matchByte = *(data - reps[0]);
- position++;
posState = (position & p->pbMask);
/*
@@ -1391,19 +1485,31 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
< MATCH [ : LIT : REP_0 ]
*/
- curPrice = curOpt->price;
- litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
+ {
+ UInt32 curPrice = curOpt->price;
+ unsigned prob = p->isMatch[state][posState];
+ matchPrice = curPrice + GET_PRICE_1(prob);
+ litPrice = curPrice + GET_PRICE_0(prob);
+ }
nextOpt = &p->opt[(size_t)cur + 1];
nextIsLit = False;
- // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
+ // here we can allow skip_items in p->opt, if we don't check (nextOpt->price < kInfinityPrice)
+ // 18.new.06
+ if ((nextOpt->price < kInfinityPrice
+ // && !IsLitState(state)
+ && matchByte == curByte)
+ || litPrice > nextOpt->price
+ )
+ litPrice = 0;
+ else
{
const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
litPrice += (!IsLitState(state) ?
LitEnc_Matched_GetPrice(probs, curByte, matchByte, p->ProbPrices) :
LitEnc_GetPrice(probs, curByte, p->ProbPrices));
-
+
if (litPrice < nextOpt->price)
{
nextOpt->price = litPrice;
@@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
- matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
-
+
+ numAvailFull = p->numAvail;
+ {
+ unsigned temp = kNumOpts - 1 - cur;
+ if (numAvailFull > temp)
+ numAvailFull = (UInt32)temp;
+ }
+
+ // 18.06
// ---------- SHORT_REP ----------
- // if (IsLitState(state)) // 18.new
+ if (IsLitState(state)) // 18.new
if (matchByte == curByte)
- // if (repMatchPrice < nextOpt->price) // 18.new
- if (nextOpt->len < 2
+ if (repMatchPrice < nextOpt->price) // 18.new
+ // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
+ if (
+ // nextOpt->price >= kInfinityPrice ||
+ nextOpt->len < 2 // we can check nextOpt->len, if skip items are not allowed in p->opt
|| (nextOpt->dist != 0
- && nextOpt->extra <= 1 // 17.old
- ))
+ // && nextOpt->extra <= 1 // 17.old
+ )
+ )
{
UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state, posState);
- if (shortRepPrice <= nextOpt->price) // 17.old
- // if (shortRepPrice < nextOpt->price) // 18.new
+ // if (shortRepPrice <= nextOpt->price) // 17.old
+ if (shortRepPrice < nextOpt->price) // 18.new
{
nextOpt->price = shortRepPrice;
nextOpt->len = 1;
@@ -1435,13 +1552,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
nextIsLit = False;
}
}
-
- numAvailFull = p->numAvail;
- {
- UInt32 temp = kNumOpts - 1 - cur;
- if (numAvailFull > temp)
- numAvailFull = temp;
- }
if (numAvailFull < 2)
continue;
@@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
// ---------- LIT : REP_0 ----------
- if (
- // litPrice != 0 && // 18.new
- !nextIsLit
+ if (!nextIsLit
+ && litPrice != 0 // 18.new
&& matchByte != curByte
&& numAvailFull > 2)
{
@@ -1465,32 +1574,32 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (limit > numAvailFull)
limit = numAvailFull;
for (len = 3; len < limit && data[len] == data2[len]; len++)
- {
- }
-
+ {}
+
{
unsigned state2 = kLiteralNextStates[state];
unsigned posState2 = (position + 1) & p->pbMask;
UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
{
unsigned offset = cur + len;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
-
+
+ if (last < offset)
+ last = offset;
+
// do
{
UInt32 price2;
COptimal *opt;
len--;
// price2 = price + GetPrice_Len_Rep_0(p, len, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len;
+ opt->len = (UInt32)len;
opt->dist = 0;
opt->extra = 1;
}
@@ -1500,8 +1609,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
-
+
startLen = 2; /* speed optimization */
+
{
// ---------- REP ----------
unsigned repIndex = 0; // 17.old
@@ -1513,31 +1623,35 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
const Byte *data2 = data - reps[repIndex];
if (data[0] != data2[0] || data[1] != data2[1])
continue;
-
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
-
+
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
+
// if (len < startLen) continue; // 18.new: speed optimization
- while (last < cur + len)
- p->opt[++last].price = kInfinityPrice;
+ {
+ unsigned offset = cur + len;
+ if (last < offset)
+ last = offset;
+ }
{
unsigned len2 = len;
price = repMatchPrice + GetPrice_PureRep(p, repIndex, state, posState);
do
{
- UInt32 price2 = price + p->repLenEnc.prices[posState][(size_t)len2 - 2];
+ UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState, len2);
COptimal *opt = &p->opt[cur + len2];
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
- opt->dist = repIndex;
+ opt->len = (UInt32)len2;
+ opt->dist = (UInt32)repIndex;
opt->extra = 0;
}
}
while (--len2 >= 2);
}
-
+
if (repIndex == 0) startLen = len + 1; // 17.old
// startLen = len + 1; // 18.new
@@ -1550,51 +1664,59 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
unsigned limit = len2 + p->numFastBytes;
if (limit > numAvailFull)
limit = numAvailFull;
-
- for (; len2 < limit && data[len2] == data2[len2]; len2++);
-
- len2 -= len;
- if (len2 >= 3)
+
+ len2 += 2;
+ if (len2 <= limit)
+ if (data[len2 - 2] == data2[len2 - 2])
+ if (data[len2 - 1] == data2[len2 - 1])
{
unsigned state2 = kRepNextStates[state];
unsigned posState2 = (position + len) & p->pbMask;
- price +=
- p->repLenEnc.prices[posState][(size_t)len - 2]
+ price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
+ GET_PRICE_0(p->isMatch[state2][posState2])
+ LitEnc_Matched_GetPrice(LIT_PROBS(position + len, data[(size_t)len - 1]),
data[len], data2[len], p->ProbPrices);
-
+
// state2 = kLiteralNextStates[state2];
state2 = kState_LitAfterRep;
posState2 = (posState2 + 1) & p->pbMask;
price += GetPrice_Rep_0(p, state2, posState2);
+
+ for (; len2 < limit && data[len2] == data2[len2]; len2++)
+ {}
+
+ len2 -= len;
+ // if (len2 >= 3)
+ {
{
unsigned offset = cur + len + len2;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
- unsigned price2;
+ UInt32 price2;
COptimal *opt;
len2--;
// price2 = price + GetPrice_Len_Rep_0(p, len2, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len2 - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len2);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
+ opt->len = (UInt32)len2;
opt->extra = (CExtra)(len + 1);
- opt->dist = repIndex;
+ opt->dist = (UInt32)repIndex;
}
}
// while (len2 >= 3);
}
}
+ }
}
}
}
@@ -1606,49 +1728,57 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
{
newLen = numAvail;
for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
- matches[numPairs] = newLen;
+ matches[numPairs] = (UInt32)newLen;
numPairs += 2;
}
-
+
+ // startLen = 2; /* speed optimization */
+
if (newLen >= startLen)
{
UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[state]);
UInt32 dist;
unsigned offs, posSlot, len;
- while (last < cur + newLen)
- p->opt[++last].price = kInfinityPrice;
+
+ {
+ unsigned offset = cur + newLen;
+ if (last < offset)
+ last = offset;
+ }
offs = 0;
while (startLen > matches[offs])
offs += 2;
dist = matches[(size_t)offs + 1];
-
+
// if (dist >= kNumFullDistances)
GetPosSlot2(dist, posSlot);
-
+
for (len = /*2*/ startLen; ; len++)
{
- UInt32 price = normalMatchPrice + p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
+ UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc, posState, len);
{
COptimal *opt;
- unsigned lenToPosState = len - 2; lenToPosState = GetLenToPosState2(lenToPosState);
+ unsigned lenNorm = len - 2;
+ lenNorm = GetLenToPosState2(lenNorm);
if (dist < kNumFullDistances)
- price += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
+ price += p->distancesPrices[lenNorm][dist & (kNumFullDistances - 1)];
else
- price += p->posSlotPrices[lenToPosState][posSlot] + p->alignPrices[dist & kAlignMask];
-
+ price += p->posSlotPrices[lenNorm][posSlot] + p->alignPrices[dist & kAlignMask];
+
opt = &p->opt[cur + len];
if (price < opt->price)
{
opt->price = price;
- opt->len = len;
+ opt->len = (UInt32)len;
opt->dist = dist + LZMA_NUM_REPS;
opt->extra = 0;
}
}
- if (/*_maxMode && */ len == matches[offs])
+ if (len == matches[offs])
{
+ // if (p->_maxMode) {
// MATCH : LIT : REP_0
const Byte *data2 = data - dist - 1;
@@ -1656,12 +1786,18 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
unsigned limit = len2 + p->numFastBytes;
if (limit > numAvailFull)
limit = numAvailFull;
-
- for (; len2 < limit && data[len2] == data2[len2]; len2++);
-
+
+ len2 += 2;
+ if (len2 <= limit)
+ if (data[len2 - 2] == data2[len2 - 2])
+ if (data[len2 - 1] == data2[len2 - 1])
+ {
+ for (; len2 < limit && data[len2] == data2[len2]; len2++)
+ {}
+
len2 -= len;
-
- if (len2 >= 3)
+
+ // if (len2 >= 3)
{
unsigned state2 = kMatchNextStates[state];
unsigned posState2 = (position + len) & p->pbMask;
@@ -1677,28 +1813,31 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price += GetPrice_Rep_0(p, state2, posState2);
offset = cur + len + len2;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
UInt32 price2;
COptimal *opt;
len2--;
// price2 = price + GetPrice_Len_Rep_0(p, len2, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len2 - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len2);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
+ opt->len = (UInt32)len2;
opt->extra = (CExtra)(len + 1);
opt->dist = dist + LZMA_NUM_REPS;
}
}
// while (len2 >= 3);
}
-
+
+ }
+
offs += 2;
if (offs == numPairs)
break;
@@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
+
+ do
+ p->opt[last].price = kInfinityPrice;
+ while (--last);
+
+ return Backward(p, cur);
}
@@ -1735,21 +1880,23 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
p->backRes = MARK_LIT;
if (numAvail < 2)
return 1;
+ // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
if (numAvail > LZMA_MATCH_LEN_MAX)
numAvail = LZMA_MATCH_LEN_MAX;
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
repLen = repIndex = 0;
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned len;
const Byte *data2 = data - p->reps[i];
if (data[0] != data2[0] || data[1] != data2[1])
continue;
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
if (len >= p->numFastBytes)
{
- p->backRes = i;
+ p->backRes = (UInt32)i;
MOVE_POS(p, len - 1)
return len;
}
@@ -1768,7 +1915,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
}
mainDist = 0; /* for GCC */
-
+
if (mainLen >= 2)
{
mainDist = p->matches[(size_t)numPairs - 1];
@@ -1793,18 +1940,18 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
|| (repLen + 2 >= mainLen && mainDist >= (1 << 9))
|| (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
{
- p->backRes = repIndex;
+ p->backRes = (UInt32)repIndex;
MOVE_POS(p, repLen - 1)
return repLen;
}
-
+
if (mainLen < 2 || numAvail <= 2)
return 1;
{
unsigned len1 = ReadMatchDistances(p, &p->numPairs);
p->longestMatchLen = len1;
-
+
if (len1 >= 2)
{
UInt32 newDist = p->matches[(size_t)p->numPairs - 1];
@@ -1815,9 +1962,9 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
return 1;
}
}
-
+
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned len, limit;
@@ -1833,7 +1980,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
break;
}
}
-
+
p->backRes = mainDist + LZMA_NUM_REPS;
if (mainLen != 2)
{
@@ -1859,7 +2006,7 @@ static void WriteEndMarker(CLzmaEnc *p, unsigned posState)
RC_BIT_0(&p->rc, prob)
}
p->state = kMatchNextStates[p->state];
-
+
p->rc.range = range;
LenEnc_Encode(&p->lenProbs, &p->rc, 0, posState);
range = p->rc.range;
@@ -1888,7 +2035,7 @@ static void WriteEndMarker(CLzmaEnc *p, unsigned posState)
}
while (--numBits);
}
-
+
{
// RcTree_ReverseEncode(&p->rc, p->posAlignEncoder, kNumAlignBits, kAlignMask);
CLzmaProb *probs = p->posAlignEncoder;
@@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p, UInt32 nowPos)
}
-
-static void FillAlignPrices(CLzmaEnc *p)
+MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
{
unsigned i;
const CProbPrice *ProbPrices = p->ProbPrices;
const CLzmaProb *probs = p->posAlignEncoder;
- p->alignPriceCount = 0;
+ // p->alignPriceCount = 0;
for (i = 0; i < kAlignTableSize / 2; i++)
{
UInt32 price = 0;
- unsigned symbol = i;
+ unsigned sym = i;
unsigned m = 1;
unsigned bit;
UInt32 prob;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
prob = probs[m];
p->alignPrices[i ] = price + GET_PRICEa_0(prob);
p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
@@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
}
-static void FillDistancesPrices(CLzmaEnc *p)
+MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
{
+ // int y; for (y = 0; y < 100; y++) {
+
UInt32 tempPrices[kNumFullDistances];
- unsigned i, lenToPosState;
+ unsigned i, lps;
const CProbPrice *ProbPrices = p->ProbPrices;
p->matchPriceCount = 0;
- for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
+ for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
{
unsigned posSlot = GetPosSlot1(i);
- unsigned footerBits = ((posSlot >> 1) - 1);
+ unsigned footerBits = (posSlot >> 1) - 1;
unsigned base = ((2 | (posSlot & 1)) << footerBits);
+ const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
// tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base, footerBits, i - base, p->ProbPrices);
-
- const CLzmaProb *probs = p->posEncoders + base;
UInt32 price = 0;
unsigned m = 1;
- unsigned symbol = i - base;
+ unsigned sym = i;
+ unsigned offset = (unsigned)1 << footerBits;
+ base += i;
+
+ if (footerBits)
do
{
- unsigned bit = symbol & 1;
- symbol >>= 1;
+ unsigned bit = sym & 1;
+ sym >>= 1;
price += GET_PRICEa(probs[m], bit);
m = (m << 1) + bit;
}
while (--footerBits);
- tempPrices[i] = price;
+
+ {
+ unsigned prob = probs[m];
+ tempPrices[base ] = price + GET_PRICEa_0(prob);
+ tempPrices[base + offset] = price + GET_PRICEa_1(prob);
+ }
}
- for (lenToPosState = 0; lenToPosState < kNumLenToPosStates; lenToPosState++)
+ for (lps = 0; lps < kNumLenToPosStates; lps++)
{
- unsigned posSlot;
- const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
- UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
- unsigned distTableSize = p->distTableSize;
- const CLzmaProb *probs = encoder;
- for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
+ unsigned slot;
+ unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
+ UInt32 *posSlotPrices = p->posSlotPrices[lps];
+ const CLzmaProb *probs = p->posSlotEncoder[lps];
+
+ for (slot = 0; slot < distTableSize2; slot++)
{
- // posSlotPrices[posSlot] = RcTree_GetPrice(encoder, kNumPosSlotBits, posSlot, p->ProbPrices);
- UInt32 price = 0;
+ // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits, slot, p->ProbPrices);
+ UInt32 price;
unsigned bit;
- unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
- UInt32 prob;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
- posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
- posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
+ unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
+ unsigned prob;
+ bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
+ posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
+ posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
}
- for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
- posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
{
- UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
+ UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
+ for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
{
- distancesPrices[0] = posSlotPrices[0];
- distancesPrices[1] = posSlotPrices[1];
- distancesPrices[2] = posSlotPrices[2];
- distancesPrices[3] = posSlotPrices[3];
+ posSlotPrices[(size_t)slot * 2 ] += delta;
+ posSlotPrices[(size_t)slot * 2 + 1] += delta;
+ delta += ((UInt32)1 << kNumBitPriceShiftBits);
}
+ }
+
+ {
+ UInt32 *dp = p->distancesPrices[lps];
+
+ dp[0] = posSlotPrices[0];
+ dp[1] = posSlotPrices[1];
+ dp[2] = posSlotPrices[2];
+ dp[3] = posSlotPrices[3];
+
for (i = 4; i < kNumFullDistances; i += 2)
{
UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
- distancesPrices[i ] = slotPrice + tempPrices[i];
- distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
+ dp[i ] = slotPrice + tempPrices[i];
+ dp[i + 1] = slotPrice + tempPrices[i + 1];
}
}
}
+ // }
}
@@ -2037,7 +2202,7 @@ void LzmaEnc_Construct(CLzmaEnc *p)
{
RangeEnc_Construct(&p->rc);
MatchFinder_Construct(&p->matchFinderBase);
-
+
#ifndef _7ZIP_ST
MatchFinderMt_Construct(&p->matchFinderMt);
p->matchFinderMt.MatchFinder = &p->matchFinderBase;
@@ -2081,7 +2246,7 @@ void LzmaEnc_Destruct(CLzmaEnc *p, ISzAllocPtr alloc, ISzAllocPtr allocBig)
#ifndef _7ZIP_ST
MatchFinderMt_Destruct(&p->matchFinderMt, allocBig);
#endif
-
+
MatchFinder_Free(&p->matchFinderBase, allocBig);
LzmaEnc_FreeLits(p, alloc);
RangeEnc_Free(&p->rc, alloc);
@@ -2126,14 +2291,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
}
if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) != 0)
-
+
for (;;)
{
UInt32 dist;
unsigned len, posState;
UInt32 range, ttt, newBound;
CLzmaProb *probs;
-
+
if (p->fastMode)
len = GetOptimumFast(p);
else
@@ -2153,9 +2318,9 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
posState = (unsigned)nowPos32 & p->pbMask;
range = p->rc.range;
probs = &p->isMatch[p->state][posState];
-
+
RC_BIT_PRE(&p->rc, probs)
-
+
dist = p->backRes;
#ifdef SHOW_STAT2
@@ -2185,7 +2350,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
RC_BIT_1(&p->rc, probs);
probs = &p->isRep[p->state];
RC_BIT_PRE(&p->rc, probs)
-
+
if (dist < LZMA_NUM_REPS)
{
RC_BIT_1(&p->rc, probs);
@@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (len != 1)
{
LenEnc_Encode(&p->repLenProbs, &p->rc, len - LZMA_MATCH_LEN_MIN, posState);
- if (!p->fastMode)
- if (--p->repLenEnc.counters[posState] == 0)
- LenPriceEnc_UpdateTable(&p->repLenEnc, posState, &p->repLenProbs, p->ProbPrices);
-
+ --p->repLenEncCounter;
p->state = kRepNextStates[p->state];
}
}
@@ -2260,34 +2422,32 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
p->state = kMatchNextStates[p->state];
LenEnc_Encode(&p->lenProbs, &p->rc, len - LZMA_MATCH_LEN_MIN, posState);
- if (!p->fastMode)
- if (--p->lenEnc.counters[posState] == 0)
- LenPriceEnc_UpdateTable(&p->lenEnc, posState, &p->lenProbs, p->ProbPrices);
+ // --p->lenEnc.counter;
dist -= LZMA_NUM_REPS;
p->reps[3] = p->reps[2];
p->reps[2] = p->reps[1];
p->reps[1] = p->reps[0];
p->reps[0] = dist + 1;
-
+
p->matchPriceCount++;
GetPosSlot(dist, posSlot);
// RcTree_Encode_PosSlot(&p->rc, p->posSlotEncoder[GetLenToPosState(len)], posSlot);
{
- UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
+ UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
range = p->rc.range;
probs = p->posSlotEncoder[GetLenToPosState(len)];
do
{
- CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
- UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
- symbol <<= 1;
+ CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
+ UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
+ sym <<= 1;
RC_BIT(&p->rc, prob, bit);
}
- while (symbol < (1 << kNumPosSlotBits * 2));
+ while (sym < (1 << kNumPosSlotBits * 2));
p->rc.range = range;
}
-
+
if (dist >= kStartPosModelIndex)
{
unsigned footerBits = ((posSlot >> 1) - 1);
@@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (dist < kNumFullDistances)
{
unsigned base = ((2 | (posSlot & 1)) << footerBits);
- RcTree_ReverseEncode(&p->rc, p->posEncoders + base, footerBits, dist - base);
+ RcTree_ReverseEncode(&p->rc, p->posEncoders + base, footerBits, (unsigned)(dist /* - base */));
}
else
{
@@ -2331,32 +2491,49 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
bit = dist & 1; dist >>= 1; RC_BIT(&p->rc, p->posAlignEncoder + m, bit); m = (m << 1) + bit;
bit = dist & 1; RC_BIT(&p->rc, p->posAlignEncoder + m, bit);
p->rc.range = range;
- p->alignPriceCount++;
+ // p->alignPriceCount++;
}
}
}
}
}
- nowPos32 += len;
+ nowPos32 += (UInt32)len;
p->additionalOffset -= len;
-
+
if (p->additionalOffset == 0)
{
UInt32 processed;
if (!p->fastMode)
{
- if (p->matchPriceCount >= (1 << 7))
+ /*
+ if (p->alignPriceCount >= 16) // kAlignTableSize
+ FillAlignPrices(p);
+ if (p->matchPriceCount >= 128)
FillDistancesPrices(p);
- if (p->alignPriceCount >= kAlignTableSize)
+ if (p->lenEnc.counter <= 0)
+ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
+ */
+ if (p->matchPriceCount >= 64)
+ {
FillAlignPrices(p);
+ // { int y; for (y = 0; y < 100; y++) {
+ FillDistancesPrices(p);
+ // }}
+ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
+ }
+ if (p->repLenEncCounter <= 0)
+ {
+ p->repLenEncCounter = REP_LEN_COUNT;
+ LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs, p->ProbPrices);
+ }
}
-
+
if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
break;
processed = nowPos32 - startPos32;
-
+
if (maxPackSize)
{
if (processed + kNumOpts + 300 >= maxUnpackSize
@@ -2430,7 +2607,7 @@ static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32 keepWindowSize, ISzAllocPtr alloc,
p->matchFinderObj = &p->matchFinderBase;
MatchFinder_CreateVTable(&p->matchFinderBase, &p->matchFinder);
}
-
+
return SZ_OK;
}
@@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
p->optEnd = 0;
p->optCur = 0;
+
+ {
+ for (i = 0; i < kNumOpts; i++)
+ p->opt[i].price = kInfinityPrice;
+ }
+
p->additionalOffset = 0;
p->pbMask = (1 << p->pb) - 1;
p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
}
+
void LzmaEnc_InitPrices(CLzmaEnc *p)
{
if (!p->fastMode)
@@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
p->lenEnc.tableSize =
p->repLenEnc.tableSize =
p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
+
+ p->repLenEncCounter = REP_LEN_COUNT;
+
LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs, p->ProbPrices);
}
@@ -2583,7 +2770,7 @@ typedef struct
ISeqOutStream vt;
Byte *data;
SizeT rem;
- Bool overflow;
+ BoolInt overflow;
} CLzmaEnc_SeqOutStreamBuf;
static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void *data, size_t size)
@@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle pp)
}
-SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
+SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32 *unpackSize)
{
CLzmaEnc *p = (CLzmaEnc *)pp;
@@ -2644,7 +2831,7 @@ SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
return SZ_ERROR_OUTPUT_EOF;
res = LzmaEnc_CodeOneBlock(p, desiredPackSize, *unpackSize);
-
+
*unpackSize = (UInt32)(p->nowPos64 - nowPos64);
*destLen -= outStream.rem;
if (outStream.overflow)
@@ -2679,7 +2866,7 @@ static SRes LzmaEnc_Encode2(CLzmaEnc *p, ICompressProgress *progress)
}
}
}
-
+
LzmaEnc_Finish(p);
/*
@@ -2751,7 +2938,7 @@ SRes LzmaEnc_MemEncode(CLzmaEncHandle pp, Byte *dest, SizeT *destLen, const Byte
p->rc.outStream = &outStream.vt;
res = LzmaEnc_MemPrepare(pp, src, srcLen, 0, alloc, allocBig);
-
+
if (res == SZ_OK)
{
res = LzmaEnc_Encode2(p, progress);
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
index d7426d38a4..0f9560c7a6 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
@@ -1,13 +1,35 @@
HISTORY of the LZMA SDK
-----------------------
+19.00 2019-02-21
+-------------------------
+- Encryption strength for 7z archives was increased:
+ the size of random initialization vector was increased from 64-bit to 128-bit,
+ and the pseudo-random number generator was improved.
+- The bug in 7zIn.c code was fixed.
+
+
+18.06 2018-12-30
+-------------------------
+- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
+ and there are minor changes in compression ratio.
+- Some bugs were fixed.
+- The bug in 7-Zip 18.02-18.05 was fixed:
+ There was memory leak in multithreading xz decoder - XzDecMt_Decode(),
+ if xz stream contains only one block.
+- The changes for MSVS compiler makefiles:
+ - the makefiles now use "PLATFORM" macroname with values (x64, x86, arm64)
+ instead of "CPU" macroname with values (AMD64, ARM64).
+ - the makefiles by default now use static version of the run-time library.
+
+
18.05 2018-04-30
-------------------------
-- The speed for LZMA/LZMA2 compressing was increased
- by 8% for fastest/fast compression levels and
+- The speed for LZMA/LZMA2 compressing was increased
+ by 8% for fastest/fast compression levels and
by 3% for normal/maximum compression levels.
- Previous versions of 7-Zip could work incorrectly in "Large memory pages" mode in
- Windows 10 because of some BUG with "Large Pages" in Windows 10.
+ Windows 10 because of some BUG with "Large Pages" in Windows 10.
Now 7-Zip doesn't use "Large Pages" on Windows 10 up to revision 1709 (16299).
- The BUG was fixed in Lzma2Enc.c
Lzma2Enc_Encode2() function worked incorretly,
@@ -16,7 +38,7 @@ HISTORY of the LZMA SDK
18.03 beta 2018-03-04
-------------------------
-- Asm\x86\LzmaDecOpt.asm: new optimized LZMA decoder written in asm
+- Asm\x86\LzmaDecOpt.asm: new optimized LZMA decoder written in asm
for x64 with about 30% higher speed than main version of LZMA decoder written in C.
- The speed for single-thread LZMA/LZMA2 decoder written in C was increased by 3%.
- 7-Zip now can use multi-threading for 7z/LZMA2 decoding,
@@ -35,7 +57,7 @@ HISTORY of the LZMA SDK
18.00 beta 2019-01-10
-------------------------
- The BUG in xz encoder was fixed:
- There was memory leak of 16 KB for each file compressed with
+ There was memory leak of 16 KB for each file compressed with
xz compression method, if additional filter was used.
@@ -44,7 +66,7 @@ HISTORY of the LZMA SDK
- Minor speed optimization for LZMA2 (xz and 7z) multi-threading compression.
7-Zip now uses additional memory buffers for multi-block LZMA2 compression.
CPU utilization was slightly improved.
-- 7-zip now creates multi-block xz archives by default. Block size can be
+- 7-zip now creates multi-block xz archives by default. Block size can be
specified with -ms[Size]{m|g} switch.
- xz decoder now can unpack random block from multi-block xz archives.
- 7-Zip command line: @listfile now doesn't work after -- switch.
@@ -55,7 +77,7 @@ HISTORY of the LZMA SDK
17.00 beta 2017-04-29
-------------------------
-- NewHandler.h / NewHandler.cpp:
+- NewHandler.h / NewHandler.cpp:
now it redefines operator new() only for old MSVC compilers (_MSC_VER < 1900).
- C/7zTypes.h : the names of variables in interface structures were changed (vt).
- Some bugs were fixed. 7-Zip could crash in some cases.
@@ -76,51 +98,51 @@ HISTORY of the LZMA SDK
16.02 2016-05-21
-------------------------
- The BUG in 16.00 - 16.01 was fixed:
- Split Handler (SplitHandler.cpp) returned incorrect
+ Split Handler (SplitHandler.cpp) returned incorrect
total size value (kpidSize) for split archives.
16.01 2016-05-19
--------------------------
+-------------------------
- Some internal changes to reduce the number of compiler warnings.
16.00 2016-05-10
--------------------------
+-------------------------
- Some bugs were fixed.
15.12 2015-11-19
--------------------------
+-------------------------
- The BUG in C version of 7z decoder was fixed:
7zDec.c : SzDecodeLzma2()
7z decoder could mistakenly report about decoding error for some 7z archives
that use LZMA2 compression method.
- The probability to get that mistaken decoding error report was about
- one error per 16384 solid blocks for solid blocks larger than 16 KB (compressed size).
+ The probability to get that mistaken decoding error report was about
+ one error per 16384 solid blocks for solid blocks larger than 16 KB (compressed size).
- The BUG (in 9.26-15.11) in C version of 7z decoder was fixed:
7zArcIn.c : SzReadHeader2()
- 7z decoder worked incorrectly for 7z archives that contain
- empty solid blocks, that can be placed to 7z archive, if some file is
+ 7z decoder worked incorrectly for 7z archives that contain
+ empty solid blocks, that can be placed to 7z archive, if some file is
unavailable for reading during archive creation.
15.09 beta 2015-10-16
--------------------------
+-------------------------
- The BUG in LZMA / LZMA2 encoding code was fixed.
The BUG in LzFind.c::MatchFinder_ReadBlock() function.
If input data size is larger than (4 GiB - dictionary_size),
the following code worked incorrectly:
- - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding functions
- for compressing from memory to memory.
+ - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding functions
+ for compressing from memory to memory.
That BUG is not related to LZMA encoder version that works via streams.
- - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly, if
- default value of chunk size (CLzma2EncProps::blockSize) is changed
+ - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly, if
+ default value of chunk size (CLzma2EncProps::blockSize) is changed
to value larger than (4 GiB - dictionary_size).
9.38 beta 2015-01-03
--------------------------
+-------------------------
- The BUG in 9.31-9.37 was fixed:
IArchiveGetRawProps interface was disabled for 7z archives.
- The BUG in 9.26-9.36 was fixed:
@@ -128,10 +150,10 @@ HISTORY of the LZMA SDK
9.36 beta 2014-12-26
--------------------------
+-------------------------
- The BUG in command line version was fixed:
7-Zip created temporary archive in current folder during update archive
- operation, if -w{Path} switch was not specified.
+ operation, if -w{Path} switch was not specified.
The fixed 7-Zip creates temporary archive in folder that contains updated archive.
- The BUG in 9.33-9.35 was fixed:
7-Zip silently ignored file reading errors during 7z or gz archive creation,
@@ -140,14 +162,14 @@ HISTORY of the LZMA SDK
9.35 beta 2014-12-07
--------------------------
+-------------------------
- 7zr.exe now support AES encryption.
- SFX modules were added to LZMA SDK
- Some bugs were fixed.
9.21 beta 2011-04-11
--------------------------
+-------------------------
- New class FString for file names at file systems.
- Speed optimization in CRC code for big-endian CPUs.
- The BUG in Lzma2Dec.c was fixed:
@@ -155,7 +177,7 @@ HISTORY of the LZMA SDK
9.18 beta 2010-11-02
--------------------------
+-------------------------
- New small SFX module for installers (SfxSetup).
@@ -168,7 +190,7 @@ HISTORY of the LZMA SDK
9.11 beta 2010-03-15
-------------------------
- PPMd compression method support
-
+
9.09 2009-12-12
-------------------------
@@ -202,7 +224,7 @@ HISTORY of the LZMA SDK
4.61 beta 2008-11-23
-------------------------
- The bug in ANSI-C LZMA Decoder was fixed:
- If encoded stream was corrupted, decoder could access memory
+ If encoded stream was corrupted, decoder could access memory
outside of allocated range.
- Some changes in ANSI-C 7z Decoder interfaces.
- LZMA SDK is placed in the public domain.
@@ -216,7 +238,7 @@ HISTORY of the LZMA SDK
4.59 beta 2008-08-13
-------------------------
- The bug was fixed:
- LZMA Encoder in fast compression mode could access memory outside of
+ LZMA Encoder in fast compression mode could access memory outside of
allocated range in some rare cases.
@@ -229,7 +251,7 @@ HISTORY of the LZMA SDK
4.57 2007-12-12
-------------------------
-- Speed optimizations in ?++ LZMA Decoder.
+- Speed optimizations in ?++ LZMA Decoder.
- Small changes for more compatibility with some C/C++ compilers.
@@ -239,36 +261,36 @@ HISTORY of the LZMA SDK
- now it supports BCJ and BCJ2 filters
- now it supports files larger than 4 GB.
- now it supports "Last Write Time" field for files.
-- C++ code for .7z archives compressing/decompressing from 7-zip
+- C++ code for .7z archives compressing/decompressing from 7-zip
was included to LZMA SDK.
-
+
4.43 2006-06-04
-------------------------
- Small changes for more compatibility with some C/C++ compilers.
-
+
4.42 2006-05-15
-------------------------
- Small changes in .h files in ANSI-C version.
-
+
4.39 beta 2006-04-14
-------------------------
- The bug in versions 4.33b:4.38b was fixed:
- C++ version of LZMA encoder could not correctly compress
+ C++ version of LZMA encoder could not correctly compress
files larger than 2 GB with HC4 match finder (-mfhc4).
-
+
4.37 beta 2005-04-06
-------------------------
-- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was defined.
+- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was defined.
4.35 beta 2005-03-02
-------------------------
- The bug was fixed in C++ version of LZMA Decoder:
- If encoded stream was corrupted, decoder could access memory
+ If encoded stream was corrupted, decoder could access memory
outside of allocated range.
@@ -339,7 +361,7 @@ HISTORY of the LZMA SDK
4.17 2005-04-18
-------------------------
-- New example for RAM->RAM compressing/decompressing:
+- New example for RAM->RAM compressing/decompressing:
LZMA + BCJ (filter for x86 code):
- LzmaRam.h
- LzmaRam.cpp
@@ -350,11 +372,11 @@ HISTORY of the LZMA SDK
4.16 2005-03-29
-------------------------
-- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):
+- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):
If _LZMA_OUT_READ was defined, and if encoded stream was corrupted,
decoder could access memory outside of allocated range.
- Speed optimization of ANSI-C LZMA Decoder (now it's about 20% faster).
- Old version of LZMA Decoder now is in file LzmaDecodeSize.c.
+ Old version of LZMA Decoder now is in file LzmaDecodeSize.c.
LzmaDecodeSize.c can provide slightly smaller code than LzmaDecode.c
- Small speed optimization in LZMA C++ code
- filter for SPARC's code was added
@@ -369,7 +391,7 @@ HISTORY of the LZMA SDK
4.05 2004-08-25
-------------------------
-- Source code of filters for x86, IA-64, ARM, ARM-Thumb
+- Source code of filters for x86, IA-64, ARM, ARM-Thumb
and PowerPC code was included to SDK
- Some internal minor changes
@@ -381,8 +403,8 @@ HISTORY of the LZMA SDK
4.03 2004-06-18
-------------------------
-- "Benchmark" command was added. It measures compressing
- and decompressing speed and shows rating values.
+- "Benchmark" command was added. It measures compressing
+ and decompressing speed and shows rating values.
Also it checks hardware errors.
@@ -411,7 +433,7 @@ HISTORY of the LZMA SDK
HISTORY of the LZMA
-------------------
- 2001-2008: Improvements to LZMA compressing/decompressing code,
+ 2001-2008: Improvements to LZMA compressing/decompressing code,
keeping compatibility with original LZMA format
1996-2001: Development of LZMA compression format
@@ -419,6 +441,6 @@ HISTORY of the LZMA
2001-08-30: LZMA compression was added to 7-Zip
1999-01-02: First version of 7-Zip was released
-
+
End of document
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
index a3deb20943..e17cd4fe1a 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
@@ -1,4 +1,4 @@
-LZMA SDK 18.05
+LZMA SDK 19.00
--------------
LZMA SDK provides the documentation, samples, header files,
--
2.29.2.windows.3
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 2/2] MdeModulePkg Lzma: Update LZMA SDK version to 19.00
2021-01-05 6:14 [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version Liu, WeiX C
@ 2021-01-05 6:14 ` Liu, WeiX C
2021-01-08 13:54 ` 回复: [edk2-devel] [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version gaoliming
1 sibling, 0 replies; 7+ messages in thread
From: Liu, WeiX C @ 2021-01-05 6:14 UTC (permalink / raw)
To: devel; +Cc: Wei Liu, Liming Gao
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
New formal release in https://www.7-zip.org/sdk.html is 19.00.
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Wei Liu <weix.c.liu@intel.com>
---
MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt | 6 +++---
MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaArchCustomDecompressLib.inf | 6 +++---
MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf | 6 +++---
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zTypes.h | 5 +++--
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zVersion.h | 8 ++++----
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/CpuArch.h | 9 +++++----
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzFind.c | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzmaDec.c | 34 +++++++++++++++++-----------------
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-history.txt | 22 ++++++++++++++++++++++
MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-sdk.txt | 2 +-
MdeModulePkg/MdeModulePkg.ci.yaml | 4 +++-
11 files changed, 163 insertions(+), 78 deletions(-)
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt b/MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt
index 3f3895b4cf..8095b6edd2 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt
@@ -1,4 +1,4 @@
-LzmaCustomDecompressLib is based on the LZMA SDK 18.05.
-LZMA SDK 18.05 was placed in the public domain on
-2018-04-30. It was released on the
+LzmaCustomDecompressLib is based on the LZMA SDK 19.00.
+LZMA SDK 19.00 was placed in the public domain on
+2019-02-21. It was released on the
http://www.7-zip.org/sdk.html website.
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaArchCustomDecompressLib.inf b/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaArchCustomDecompressLib.inf
index d789e7dc42..3a0647e7f8 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaArchCustomDecompressLib.inf
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaArchCustomDecompressLib.inf
@@ -1,11 +1,11 @@
## @file
# LzmaArchCustomDecompressLib produces LZMA custom decompression algorithm with the converter for the different arch code.
#
-# It is based on the LZMA SDK 18.05
-# LZMA SDK 18.05 was placed in the public domain on 2018-04-30.
+# It is based on the LZMA SDK 19.00
+# LZMA SDK 19.00 was placed in the public domain on 2019-02-21.
# It was released on the http://www.7-zip.org/sdk.html website.
#
-# Copyright (c) 2012 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2012 - 2020, Intel Corporation. All rights reserved.<BR>
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf b/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf
index 4ed1d83a40..39e66136c4 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/LzmaCustomDecompressLib.inf
@@ -1,11 +1,11 @@
## @file
# LzmaCustomDecompressLib produces LZMA custom decompression algorithm.
#
-# It is based on the LZMA SDK 18.05.
-# LZMA SDK 18.05 was placed in the public domain on 2018-04-30.
+# It is based on the LZMA SDK 19.00.
+# LZMA SDK 19.00 was placed in the public domain on 2019-02-21.
# It was released on the http://www.7-zip.org/sdk.html website.
#
-# Copyright (c) 2009 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2009 - 2020, Intel Corporation. All rights reserved.<BR>
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zTypes.h b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zTypes.h
index c89b5c2433..42b7040dd7 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zTypes.h
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zTypes.h
@@ -1,5 +1,5 @@
/* 7zTypes.h -- Basic types
-2017-07-17 : Igor Pavlov : Public domain */
+2018-08-04 : Igor Pavlov : Public domain */
#ifndef __7Z_TYPES_H
#define __7Z_TYPES_H
@@ -107,7 +107,8 @@ typedef UInt32 SizeT;
typedef size_t SizeT;
#endif
-typedef int Bool;
+typedef int BoolInt;
+/* typedef BoolInt Bool; */
#define True 1
#define False 0
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zVersion.h b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zVersion.h
index ed3aa94270..0074c64be9 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zVersion.h
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/7zVersion.h
@@ -1,7 +1,7 @@
-#define MY_VER_MAJOR 18
-#define MY_VER_MINOR 05
+#define MY_VER_MAJOR 19
+#define MY_VER_MINOR 00
#define MY_VER_BUILD 0
-#define MY_VERSION_NUMBERS "18.05"
+#define MY_VERSION_NUMBERS "19.00"
#define MY_VERSION MY_VERSION_NUMBERS
#ifdef MY_CPU_NAME
@@ -10,7 +10,7 @@
#define MY_VERSION_CPU MY_VERSION
#endif
-#define MY_DATE "2018-04-30"
+#define MY_DATE "2019-02-21"
#undef MY_COPYRIGHT
#undef MY_VERSION_COPYRIGHT_DATE
#define MY_AUTHOR_NAME "Igor Pavlov"
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/CpuArch.h b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/CpuArch.h
index 7fb27282c7..5f74c1c0cb 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/CpuArch.h
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/CpuArch.h
@@ -1,5 +1,5 @@
/* CpuArch.h -- CPU specific code
-2017-09-04 : Igor Pavlov : Public domain */
+2018-02-18 : Igor Pavlov : Public domain */
#ifndef __CPU_ARCH_H
#define __CPU_ARCH_H
@@ -318,15 +318,16 @@ enum
void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d);
-Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
+BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
int x86cpuid_GetFirm(const Cx86cpuid *p);
#define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) & 0xF))
#define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) & 0xF))
#define x86cpuid_GetStepping(ver) (ver & 0xF)
-Bool CPU_Is_InOrder();
-Bool CPU_Is_Aes_Supported();
+BoolInt CPU_Is_InOrder();
+BoolInt CPU_Is_Aes_Supported();
+BoolInt CPU_IsSupported_PageGB();
#endif
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzFind.c b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzFind.c
index 8765cbebfc..670a9fab12 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzFind.c
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzFind.c
@@ -1,5 +1,5 @@
/* LzFind.c -- Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-07-08 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -140,7 +140,7 @@ static void MatchFinder_SetDefaultSettings(CMatchFinder *p)
void MatchFinder_Construct(CMatchFinder *p)
{
- UInt32 i;
+ unsigned i;
p->bufferBase = NULL;
p->directInput = 0;
p->hash = NULL;
@@ -149,7 +149,7 @@ void MatchFinder_Construct(CMatchFinder *p)
for (i = 0; i < 256; i++)
{
- UInt32 r = i;
+ UInt32 r = (UInt32)i;
unsigned j;
for (j = 0; j < 8; j++)
r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
@@ -370,6 +370,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
MatchFinder_ReduceOffsets(p, subValue);
}
+
+MY_NO_INLINE
static void MatchFinder_CheckLimits(CMatchFinder *p)
{
if (p->pos == kMaxValForNormalize)
@@ -381,10 +383,16 @@ static void MatchFinder_CheckLimits(CMatchFinder *p)
MatchFinder_SetLimits(p);
}
-static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
+
+/*
+ (lenLimit > maxLen)
+*/
+MY_FORCE_INLINE
+static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
- UInt32 *distances, UInt32 maxLen)
+ UInt32 *distances, unsigned maxLen)
{
+ /*
son[_cyclicBufferPos] = curMatch;
for (;;)
{
@@ -402,7 +410,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = len;
*distances++ = delta - 1;
if (len == lenLimit)
return distances;
@@ -410,15 +419,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
}
}
}
+ */
+
+ const Byte *lim = cur + lenLimit;
+ son[_cyclicBufferPos] = curMatch;
+ do
+ {
+ UInt32 delta = pos - curMatch;
+ if (delta >= _cyclicBufferSize)
+ break;
+ {
+ ptrdiff_t diff;
+ curMatch = son[_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)];
+ diff = (ptrdiff_t)0 - delta;
+ if (cur[maxLen] == cur[maxLen + diff])
+ {
+ const Byte *c = cur;
+ while (*c == c[diff])
+ {
+ if (++c == lim)
+ {
+ distances[0] = (UInt32)(lim - cur);
+ distances[1] = delta - 1;
+ return distances + 2;
+ }
+ }
+ {
+ unsigned len = (unsigned)(c - cur);
+ if (maxLen < len)
+ {
+ maxLen = len;
+ distances[0] = (UInt32)len;
+ distances[1] = delta - 1;
+ distances += 2;
+ }
+ }
+ }
+ }
+ }
+ while (--cutValue);
+
+ return distances;
}
+
+MY_FORCE_INLINE
UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
UInt32 *distances, UInt32 maxLen)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -428,9 +480,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
return distances;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = pair[0];
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -439,11 +492,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = (UInt32)len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
+ *ptr1 = pair0;
*ptr0 = pair[1];
return distances;
}
@@ -470,9 +524,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -482,9 +536,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
return;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
if (pb[len] == cur[len])
{
while (++len != lenLimit)
@@ -522,13 +576,13 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
p->buffer++; \
if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
-#define MOVE_POS_RET MOVE_POS return offset;
+#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define GET_MATCHES_HEADER2(minLen, ret_op) \
- UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
- lenLimit = p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
+ unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
+ lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
cur = p->buffer;
#define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen, return 0)
@@ -537,22 +591,22 @@ static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos, p->cyclicBufferSize, p->cutValue
#define GET_MATCHES_FOOTER(offset, maxLen) \
- offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
- distances + offset, maxLen) - distances); MOVE_POS_RET;
+ offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch, MF_PARAMS(p), \
+ distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
#define SKIP_FOOTER \
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
#define UPDATE_maxLen { \
ptrdiff_t diff = (ptrdiff_t)0 - d2; \
const Byte *c = cur + maxLen; \
const Byte *lim = cur + lenLimit; \
for (; c != lim; c++) if (*(c + diff) != *c) break; \
- maxLen = (UInt32)(c - cur); }
+ maxLen = (unsigned)(c - cur); }
static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(2)
HASH2_CALC;
curMatch = p->hash[hv];
@@ -563,7 +617,7 @@ static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
@@ -574,7 +628,8 @@ UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, d2, maxLen, offset, pos;
+ UInt32 h2, d2, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(3)
@@ -596,12 +651,12 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
UPDATE_maxLen
- distances[0] = maxLen;
+ distances[0] = (UInt32)maxLen;
distances[1] = d2 - 1;
offset = 2;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
@@ -611,7 +666,8 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -634,7 +690,8 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
@@ -650,10 +707,10 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
@@ -743,7 +800,8 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -766,7 +824,8 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
@@ -782,7 +841,7 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
p->son[p->cyclicBufferPos] = curMatch;
@@ -793,7 +852,7 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (maxLen < 3)
maxLen = 3;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances + offset, maxLen) - (distances));
MOVE_POS_RET
}
@@ -879,12 +938,12 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
p->hash[hv] = p->pos;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances, 2) - (distances));
MOVE_POS_RET
}
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzmaDec.c b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzmaDec.c
index a262319928..80fb74525a 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzmaDec.c
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/C/LzmaDec.c
@@ -1,15 +1,15 @@
/* LzmaDec.c -- LZMA Decoder
-2018-02-28 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
-/* #include "CpuArch.h" */
-#include "LzmaDec.h"
-
#ifndef EFIAPI
#include <string.h>
#endif
+/* #include "CpuArch.h" */
+#include "LzmaDec.h"
+
#define kNumTopBits 24
#define kTopValue ((UInt32)1 << kNumTopBits)
@@ -21,7 +21,7 @@
#define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
#define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt - (ttt >> kNumMoveBits));
#define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
@@ -68,7 +68,7 @@
#define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit) return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0_CHECK range = bound;
#define UPDATE_1_CHECK range -= bound; code -= bound;
#define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
@@ -541,7 +541,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
curLen = ((rem < len) ? (unsigned)rem : len);
pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
- processedPos += curLen;
+ processedPos += (UInt32)curLen;
len -= curLen;
if (curLen <= dicBufSize - pos)
@@ -549,7 +549,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
Byte *dest = dic + dicPos;
ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
const Byte *lim = dest + curLen;
- dicPos += curLen;
+ dicPos += (SizeT)curLen;
do
*(dest) = (Byte)*(dest + src);
while (++dest != lim);
@@ -574,14 +574,14 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
p->buf = buf;
p->range = range;
p->code = code;
- p->remainLen = len;
+ p->remainLen = (UInt32)len;
p->dicPos = dicPos;
p->processedPos = processedPos;
p->reps[0] = rep0;
p->reps[1] = rep1;
p->reps[2] = rep2;
p->reps[3] = rep3;
- p->state = state;
+ p->state = (UInt32)state;
return SZ_OK;
}
@@ -603,8 +603,8 @@ static void MY_FAST_CALL LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
p->checkDicSize = p->prop.dicSize;
- p->processedPos += len;
- p->remainLen -= len;
+ p->processedPos += (UInt32)len;
+ p->remainLen -= (UInt32)len;
while (len != 0)
{
len--;
@@ -852,7 +852,7 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, const Byte *buf, SizeT inS
}
-void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
+void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt initState)
{
p->remainLen = kMatchSpecLenStart + 1;
p->tempBufSize = 0;
@@ -981,10 +981,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
p->tempBufSize = rem;
if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
{
- int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
+ int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, (SizeT)rem);
if (dummyRes == DUMMY_ERROR)
{
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
*status = LZMA_STATUS_NEEDS_MORE_INPUT;
return SZ_OK;
}
@@ -1007,9 +1007,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
return SZ_ERROR_FAIL; /* some internal error */
lookAhead -= rem;
}
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
src += lookAhead;
- inSize -= lookAhead;
+ inSize -= (SizeT)lookAhead;
p->tempBufSize = 0;
}
}
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-history.txt b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-history.txt
index 5ad9a5b1b9..96090b2b45 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-history.txt
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-history.txt
@@ -1,6 +1,28 @@
HISTORY of the LZMA SDK
-----------------------
+19.00 2019-02-21
+-------------------------
+- Encryption strength for 7z archives was increased:
+ the size of random initialization vector was increased from 64-bit to 128-bit,
+ and the pseudo-random number generator was improved.
+- The bug in 7zIn.c code was fixed.
+
+
+18.06 2018-12-30
+-------------------------
+- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
+ and there are minor changes in compression ratio.
+- Some bugs were fixed.
+- The bug in 7-Zip 18.02-18.05 was fixed:
+ There was memory leak in multithreading xz decoder - XzDecMt_Decode(),
+ if xz stream contains only one block.
+- The changes for MSVS compiler makefiles:
+ - the makefiles now use "PLATFORM" macroname with values (x64, x86, arm64)
+ instead of "CPU" macroname with values (AMD64, ARM64).
+ - the makefiles by default now use static version of the run-time library.
+
+
18.05 2018-04-30
-------------------------
- The speed for LZMA/LZMA2 compressing was increased
diff --git a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-sdk.txt b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-sdk.txt
index bf0b3a5ee6..6bb6fa194e 100644
--- a/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-sdk.txt
+++ b/MdeModulePkg/Library/LzmaCustomDecompressLib/Sdk/DOC/lzma-sdk.txt
@@ -1,4 +1,4 @@
-LZMA SDK 18.05
+LZMA SDK 19.00
--------------
LZMA SDK provides the documentation, samples, header files,
diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml b/MdeModulePkg/MdeModulePkg.ci.yaml
index 20d53fc5a5..45783f12c1 100644
--- a/MdeModulePkg/MdeModulePkg.ci.yaml
+++ b/MdeModulePkg/MdeModulePkg.ci.yaml
@@ -20,7 +20,9 @@
## Both file path and directory path are accepted.
"IgnoreFiles": [
"Library/BrotliCustomDecompressLib/brotli",
- "Universal/RegularExpressionDxe/oniguruma"
+ "Universal/RegularExpressionDxe/oniguruma",
+ "Library/LzmaCustomDecompressLib/Sdk/DOC",
+ "Library/LzmaCustomDecompressLib/Sdk/C"
]
},
## options defined ci/Plugin/CompilerPlugin
--
2.29.2.windows.3
^ permalink raw reply related [flat|nested] 7+ messages in thread* 回复: [edk2-devel] [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
2021-01-05 6:14 [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version Liu, WeiX C
2021-01-05 6:14 ` [PATCH 2/2] MdeModulePkg Lzma: Update LZMA SDK version to 19.00 Liu, WeiX C
@ 2021-01-08 13:54 ` gaoliming
1 sibling, 0 replies; 7+ messages in thread
From: gaoliming @ 2021-01-08 13:54 UTC (permalink / raw)
To: devel, weix.c.liu
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Merged https://github.com/tianocore/edk2/pull/1272
Thanks
Liming
> -----邮件原件-----
> 发件人: bounce+27952+69675+4905953+8761045@groups.io
> <bounce+27952+69675+4905953+8761045@groups.io> 代表 Liu, WeiX C
> 发送时间: 2021年1月5日 14:14
> 收件人: devel@edk2.groups.io
> 抄送: Wei Liu <weix.c.liu@intel.com>; Liming Gao
> <gaoliming@byosoft.com.cn>
> 主题: [edk2-devel] [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to
> new 19.00 version
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
> New formal release in https://www.7-zip.org/sdk.html is 19.00.
>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Signed-off-by: Wei Liu <weix.c.liu@intel.com>
> ---
> BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4
> ++--
> BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5
> +++--
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9
> +++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 232
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
+++++++++++++++++++++-------------------------------------------------------
-------
> -------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14
> +++++++-------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32
> ++++++++++++++++----------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 907
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++++++++++++++++++++++++++++++++++++++++-----------------
>
----------------------------------------------------------------------------
-----------------
>
----------------------------------------------------------------------------
-----------------
>
----------------------------------------------------------------------------
-----------------
> ----------------------------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 114
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++----------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
> 12 files changed, 877 insertions(+), 575 deletions(-)
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> index 7cf967a774..b2338e2598 100644
> --- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> +++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> @@ -1,3 +1,3 @@
> -LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
>
> -was placed in the public domain on 2018-04-30. It was
>
> +LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
>
> +was placed in the public domain on 2019-02-21. It was
>
> released on the http://www.7-zip.org/sdk.html website.
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> index bebdb9aa84..bf4706ee23 100644
> --- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> +++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> @@ -1,9 +1,9 @@
> /** @file
>
> LZMA Compress/Decompress tool (LzmaCompress)
>
>
>
> - Based on LZMA SDK 18.05:
>
> + Based on LZMA SDK 19.00:
>
> LzmaUtil.c -- Test application for LZMA compression
>
> - 2018-04-30 : Igor Pavlov : Public domain
>
> + 2019-02-21 : Igor Pavlov : Public domain
>
>
>
> Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
>
> @@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate
> memory";
> const char *kDataErrorMessage = "Data error";
>
> const char *kInvalidParamValMessage = "Invalid parameter value";
>
>
>
> -static Bool mQuietMode = False;
>
> +static BoolInt mQuietMode = False;
>
> static CONVERTER_TYPE mConType = NoConverter;
>
>
>
> UINT64 mDictionarySize = 28;
>
> @@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
> CFileOutStream outStream;
>
> int res;
>
> int encodeMode = 0;
>
> - Bool modeWasSet = False;
>
> + BoolInt modeWasSet = False;
>
> const char *inputFile = NULL;
>
> const char *outputFile = "file.tmp";
>
> int param;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> index 4977cdaa66..593f5aa259 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> @@ -1,5 +1,5 @@
> /* 7zTypes.h -- Basic types
>
> -2017-07-17 : Igor Pavlov : Public domain */
>
> +2018-08-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __7Z_TYPES_H
>
> #define __7Z_TYPES_H
>
> @@ -103,7 +103,8 @@ typedef UInt32 SizeT;
> typedef size_t SizeT;
>
> #endif
>
>
>
> -typedef int Bool;
>
> +typedef int BoolInt;
>
> +/* typedef BoolInt Bool; */
>
> #define True 1
>
> #define False 0
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> index ed3aa94270..0074c64be9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> @@ -1,7 +1,7 @@
> -#define MY_VER_MAJOR 18
>
> -#define MY_VER_MINOR 05
>
> +#define MY_VER_MAJOR 19
>
> +#define MY_VER_MINOR 00
>
> #define MY_VER_BUILD 0
>
> -#define MY_VERSION_NUMBERS "18.05"
>
> +#define MY_VERSION_NUMBERS "19.00"
>
> #define MY_VERSION MY_VERSION_NUMBERS
>
>
>
> #ifdef MY_CPU_NAME
>
> @@ -10,7 +10,7 @@
> #define MY_VERSION_CPU MY_VERSION
>
> #endif
>
>
>
> -#define MY_DATE "2018-04-30"
>
> +#define MY_DATE "2019-02-21"
>
> #undef MY_COPYRIGHT
>
> #undef MY_VERSION_COPYRIGHT_DATE
>
> #define MY_AUTHOR_NAME "Igor Pavlov"
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> index 7fb27282c7..5f74c1c0cb 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> @@ -1,5 +1,5 @@
> /* CpuArch.h -- CPU specific code
>
> -2017-09-04 : Igor Pavlov : Public domain */
>
> +2018-02-18 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __CPU_ARCH_H
>
> #define __CPU_ARCH_H
>
> @@ -318,15 +318,16 @@ enum
>
>
> void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32
*d);
>
>
>
> -Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> +BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> int x86cpuid_GetFirm(const Cx86cpuid *p);
>
>
>
> #define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) &
0xF))
>
> #define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) &
> 0xF))
>
> #define x86cpuid_GetStepping(ver) (ver & 0xF)
>
>
>
> -Bool CPU_Is_InOrder();
>
> -Bool CPU_Is_Aes_Supported();
>
> +BoolInt CPU_Is_InOrder();
>
> +BoolInt CPU_Is_Aes_Supported();
>
> +BoolInt CPU_IsSupported_PageGB();
>
>
>
> #endif
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> index 6ea82a9b53..f89c9ddd7d 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> @@ -1,5 +1,5 @@
> /* LzFind.c -- Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-07-08 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -73,7 +73,7 @@ static void MatchFinder_ReadBlock(CMatchFinder *p)
> p->streamEndWasReached = 1;
>
> return;
>
> }
>
> -
>
> +
>
> for (;;)
>
> {
>
> Byte *dest = p->buffer + (p->streamPos - p->pos);
>
> @@ -138,7 +138,7 @@ static void
> MatchFinder_SetDefaultSettings(CMatchFinder *p)
>
>
> void MatchFinder_Construct(CMatchFinder *p)
>
> {
>
> - UInt32 i;
>
> + unsigned i;
>
> p->bufferBase = NULL;
>
> p->directInput = 0;
>
> p->hash = NULL;
>
> @@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
>
>
> for (i = 0; i < 256; i++)
>
> {
>
> - UInt32 r = i;
>
> + UInt32 r = (UInt32)i;
>
> unsigned j;
>
> for (j = 0; j < 8; j++)
>
> r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
>
> @@ -180,24 +180,24 @@ int MatchFinder_Create(CMatchFinder *p, UInt32
> historySize,
> ISzAllocPtr alloc)
>
> {
>
> UInt32 sizeReserv;
>
> -
>
> +
>
> if (historySize > kMaxHistorySize)
>
> {
>
> MatchFinder_Free(p, alloc);
>
> return 0;
>
> }
>
> -
>
> +
>
> sizeReserv = historySize >> 1;
>
> if (historySize >= ((UInt32)3 << 30)) sizeReserv = historySize >>
3;
>
> else if (historySize >= ((UInt32)2 << 30)) sizeReserv = historySize >>
2;
>
> -
>
> +
>
> sizeReserv += (keepAddBufferBefore + matchMaxLen +
> keepAddBufferAfter) / 2 + (1 << 19);
>
>
>
> p->keepSizeBefore = historySize + keepAddBufferBefore + 1;
>
> p->keepSizeAfter = matchMaxLen + keepAddBufferAfter;
>
> -
>
> +
>
> /* we need one additional byte, since we use MoveBlock after pos++ and
> before dictionary using */
>
> -
>
> +
>
> if (LzInWindow_Create(p, sizeReserv, alloc))
>
> {
>
> UInt32 newCyclicBufferSize = historySize + 1;
>
> @@ -243,7 +243,7 @@ int MatchFinder_Create(CMatchFinder *p, UInt32
> historySize,
> p->historySize = historySize;
>
> p->hashSizeSum = hs;
>
> p->cyclicBufferSize = newCyclicBufferSize;
>
> -
>
> +
>
> numSons = newCyclicBufferSize;
>
> if (p->btMode)
>
> numSons <<= 1;
>
> @@ -251,11 +251,11 @@ int MatchFinder_Create(CMatchFinder *p, UInt32
> historySize,
>
>
> if (p->hash && p->numRefs == newSize)
>
> return 1;
>
> -
>
> +
>
> MatchFinder_FreeThisClassMemory(p, alloc);
>
> p->numRefs = newSize;
>
> p->hash = AllocRefs(newSize, alloc);
>
> -
>
> +
>
> if (p->hash)
>
> {
>
> p->son = p->hash + p->hashSizeSum;
>
> @@ -272,11 +272,11 @@ static void MatchFinder_SetLimits(CMatchFinder
> *p)
> {
>
> UInt32 limit = kMaxValForNormalize - p->pos;
>
> UInt32 limit2 = p->cyclicBufferSize - p->cyclicBufferPos;
>
> -
>
> +
>
> if (limit2 < limit)
>
> limit = limit2;
>
> limit2 = p->streamPos - p->pos;
>
> -
>
> +
>
> if (limit2 <= p->keepSizeAfter)
>
> {
>
> if (limit2 > 0)
>
> @@ -284,10 +284,10 @@ static void MatchFinder_SetLimits(CMatchFinder
> *p)
> }
>
> else
>
> limit2 -= p->keepSizeAfter;
>
> -
>
> +
>
> if (limit2 < limit)
>
> limit = limit2;
>
> -
>
> +
>
> {
>
> UInt32 lenLimit = p->streamPos - p->pos;
>
> if (lenLimit > p->matchMaxLen)
>
> @@ -326,10 +326,10 @@ void MatchFinder_Init_3(CMatchFinder *p, int
> readData)
> p->streamPos = p->cyclicBufferSize;
>
> p->result = SZ_OK;
>
> p->streamEndWasReached = 0;
>
> -
>
> +
>
> if (readData)
>
> MatchFinder_ReadBlock(p);
>
> -
>
> +
>
> MatchFinder_SetLimits(p);
>
> }
>
>
>
> @@ -341,7 +341,7 @@ void MatchFinder_Init(CMatchFinder *p)
> MatchFinder_Init_3(p, True);
>
> }
>
>
>
> -
>
> +
>
> static UInt32 MatchFinder_GetSubValue(CMatchFinder *p)
>
> {
>
> return (p->pos - p->historySize - 1) & kNormalizeMask;
>
> @@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
> MatchFinder_ReduceOffsets(p, subValue);
>
> }
>
>
>
> +
>
> +MY_NO_INLINE
>
> static void MatchFinder_CheckLimits(CMatchFinder *p)
>
> {
>
> if (p->pos == kMaxValForNormalize)
>
> @@ -379,10 +381,16 @@ static void
> MatchFinder_CheckLimits(CMatchFinder *p)
> MatchFinder_SetLimits(p);
>
> }
>
>
>
> -static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> +
>
> +/*
>
> + (lenLimit > maxLen)
>
> +*/
>
> +MY_FORCE_INLINE
>
> +static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> - UInt32 *distances, UInt32 maxLen)
>
> + UInt32 *distances, unsigned maxLen)
>
> {
>
> + /*
>
> son[_cyclicBufferPos] = curMatch;
>
> for (;;)
>
> {
>
> @@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos,
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> return distances;
>
> @@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32
> lenLimit, UInt32 curMatch, UInt32 pos,
> }
>
> }
>
> }
>
> + */
>
> +
>
> + const Byte *lim = cur + lenLimit;
>
> + son[_cyclicBufferPos] = curMatch;
>
> + do
>
> + {
>
> + UInt32 delta = pos - curMatch;
>
> + if (delta >= _cyclicBufferSize)
>
> + break;
>
> + {
>
> + ptrdiff_t diff;
>
> + curMatch = son[_cyclicBufferPos - delta + ((delta >
_cyclicBufferPos) ?
> _cyclicBufferSize : 0)];
>
> + diff = (ptrdiff_t)0 - delta;
>
> + if (cur[maxLen] == cur[maxLen + diff])
>
> + {
>
> + const Byte *c = cur;
>
> + while (*c == c[diff])
>
> + {
>
> + if (++c == lim)
>
> + {
>
> + distances[0] = (UInt32)(lim - cur);
>
> + distances[1] = delta - 1;
>
> + return distances + 2;
>
> + }
>
> + }
>
> + {
>
> + unsigned len = (unsigned)(c - cur);
>
> + if (maxLen < len)
>
> + {
>
> + maxLen = len;
>
> + distances[0] = (UInt32)len;
>
> + distances[1] = delta - 1;
>
> + distances += 2;
>
> + }
>
> + }
>
> + }
>
> + }
>
> + }
>
> + while (--cutValue);
>
> +
>
> + return distances;
>
> }
>
>
>
> +
>
> +MY_FORCE_INLINE
>
> UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> UInt32 *distances, UInt32 maxLen)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> return distances;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = pair[0];
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const Byt
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = (UInt32)len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> + *ptr1 = pair0;
>
> *ptr0 = pair[1];
>
> return distances;
>
> }
>
> @@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const
> return;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> if (pb[len] == cur[len])
>
> {
>
> while (++len != lenLimit)
>
> @@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const
> p->buffer++; \
>
> if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
>
>
>
> -#define MOVE_POS_RET MOVE_POS return offset;
>
> +#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
>
>
>
> static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
>
>
>
> #define GET_MATCHES_HEADER2(minLen, ret_op) \
>
> - UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> - lenLimit = p->lenLimit; { if (lenLimit < minLen) {
MatchFinder_MovePos(p);
> ret_op; }} \
>
> + unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> + lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen)
> { MatchFinder_MovePos(p); ret_op; }} \
>
> cur = p->buffer;
>
>
>
> #define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen,
> return 0)
>
> @@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder
> *p) { MOVE_POS; }
> #define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos,
> p->cyclicBufferSize, p->cutValue
>
>
>
> #define GET_MATCHES_FOOTER(offset, maxLen) \
>
> - offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
>
> - distances + offset, maxLen) - distances); MOVE_POS_RET;
>
> + offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch,
> MF_PARAMS(p), \
>
> + distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
>
>
>
> #define SKIP_FOOTER \
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
> MOVE_POS;
>
>
>
> #define UPDATE_maxLen { \
>
> ptrdiff_t diff = (ptrdiff_t)0 - d2; \
>
> const Byte *c = cur + maxLen; \
>
> const Byte *lim = cur + lenLimit; \
>
> for (; c != lim; c++) if (*(c + diff) != *c) break; \
>
> - maxLen = (UInt32)(c - cur); }
>
> + maxLen = (unsigned)(c - cur); }
>
>
>
> static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(2)
>
> HASH2_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -561,7 +615,7 @@ static UInt32
> Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -572,7 +626,8 @@ UInt32
> Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, d2, maxLen, offset, pos;
>
> + UInt32 h2, d2, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(3)
>
>
>
> @@ -584,7 +639,7 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> d2 = pos - hash[h2];
>
>
>
> curMatch = (hash + kFix3HashSize)[hv];
>
> -
>
> +
>
> hash[h2] = pos;
>
> (hash + kFix3HashSize)[hv] = pos;
>
>
>
> @@ -594,22 +649,23 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> UPDATE_maxLen
>
> - distances[0] = maxLen;
>
> + distances[0] = (UInt32)maxLen;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> -
>
> +
>
> GET_MATCHES_FOOTER(offset, maxLen)
>
> }
>
>
>
> static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -618,25 +674,26 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
>
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> maxLen = 0;
>
> offset = 0;
>
> -
>
> +
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> -
>
> +
>
> if (d2 != d3 && d3 < p->cyclicBufferSize && *(cur - d3) == *cur)
>
> {
>
> maxLen = 3;
>
> @@ -644,21 +701,21 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> offset += 2;
>
> d2 = d3;
>
> }
>
> -
>
> +
>
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> -
>
> +
>
> if (maxLen < 3)
>
> maxLen = 3;
>
> -
>
> +
>
> GET_MATCHES_FOOTER(offset, maxLen)
>
> }
>
>
>
> @@ -674,13 +731,13 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -710,7 +767,7 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> offset = 2;
>
> d2 = d3;
>
> }
>
> -
>
> +
>
> if (d2 != d4 && d4 < p->cyclicBufferSize
>
> && *(cur - d4) == *cur
>
> && *(cur - d4 + 3) == *(cur + 3))
>
> @@ -720,7 +777,7 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> offset += 2;
>
> d2 = d4;
>
> }
>
> -
>
> +
>
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> @@ -734,14 +791,15 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (maxLen < 4)
>
> maxLen = 4;
>
> -
>
> +
>
> GET_MATCHES_FOOTER(offset, maxLen)
>
> }
>
> */
>
>
>
> static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -749,13 +807,12 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> hash = p->hash;
>
> pos = p->pos;
>
> -
>
> - d2 = pos - hash[ h2];
>
> +
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> -
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -764,11 +821,12 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> -
>
> +
>
> if (d2 != d3 && d3 < p->cyclicBufferSize && *(cur - d3) == *cur)
>
> {
>
> maxLen = 3;
>
> @@ -776,22 +834,22 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> offset += 2;
>
> d2 = d3;
>
> }
>
> -
>
> +
>
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> MOVE_POS_RET;
>
> }
>
> }
>
> -
>
> +
>
> if (maxLen < 3)
>
> maxLen = 3;
>
>
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances + offset, maxLen) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -807,14 +865,14 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> hash = p->hash;
>
> pos = p->pos;
>
> -
>
> - d2 = pos - hash[ h2];
>
> +
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -844,7 +902,7 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> offset = 2;
>
> d2 = d3;
>
> }
>
> -
>
> +
>
> if (d2 != d4 && d4 < p->cyclicBufferSize
>
> && *(cur - d4) == *cur
>
> && *(cur - d4 + 3) == *(cur + 3))
>
> @@ -854,7 +912,7 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> offset += 2;
>
> d2 = d4;
>
> }
>
> -
>
> +
>
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> @@ -865,7 +923,7 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> MOVE_POS_RET;
>
> }
>
> }
>
> -
>
> +
>
> if (maxLen < 4)
>
> maxLen = 4;
>
>
>
> @@ -877,12 +935,12 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> p->hash[hv] = p->pos;
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances, 2) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> SKIP_FOOTER
>
> @@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> @@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> index 2563824fcd..df32146f92 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> @@ -1,5 +1,5 @@
> /* LzFindMt.c -- multithreaded Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-12-29 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -232,38 +232,57 @@ static void
> MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
>
>
> #define kEmptyHashValue 0
>
>
>
> -/* #define MFMT_GM_INLINE */
>
> +#define MFMT_GM_INLINE
>
>
>
> #ifdef MFMT_GM_INLINE
>
>
>
> -#define NO_INLINE MY_FAST_CALL
>
> +/*
>
> + we use size_t for _cyclicBufferPos instead of UInt32
>
> + to eliminate "movsx" BUG in old MSVC x64 compiler.
>
> +*/
>
>
>
> -static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> - UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> - UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit,
> UInt32 size, UInt32 *posRes)
>
> +MY_NO_INLINE
>
> +static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte
> *cur, CLzRef *son,
>
> + size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> + UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32
> *limit, UInt32 size, UInt32 *posRes)
>
> {
>
> do
>
> {
>
> - UInt32 *distances = _distances + 1;
>
> - UInt32 curMatch = pos - *hash++;
>
> + UInt32 *_distances = ++distances;
>
> + UInt32 delta = *hash++;
>
>
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> UInt32 cutValue = _cutValue;
>
> - UInt32 maxLen = _maxLen;
>
> - for (;;)
>
> + unsigned maxLen = (unsigned)_maxLen;
>
> +
>
> + /*
>
> + if (size > 1)
>
> {
>
> - UInt32 delta = pos - curMatch;
>
> - if (cutValue-- == 0 || delta >= _cyclicBufferSize)
>
> + UInt32 delta = *hash;
>
> + if (delta < _cyclicBufferSize)
>
> {
>
> - *ptr0 = *ptr1 = kEmptyHashValue;
>
> - break;
>
> + UInt32 cyc1 = _cyclicBufferPos + 1;
>
> + CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ?
> _cyclicBufferSize : 0)) << 1);
>
> + Byte b = *(cur + 1 - delta);
>
> + _distances[0] = pair[0];
>
> + _distances[1] = b;
>
> }
>
> + }
>
> + */
>
> + if (cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + }
>
> + else
>
> + for(;;)
>
> + {
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta +
> ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = *pair;
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32
> lenLimit, UInt32 pos, const Byte *
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> - *ptr0 = pair[1];
>
> + UInt32 pair1 = pair[1];
>
> + *ptr1 = pair0;
>
> + *ptr0 = pair1;
>
> break;
>
> }
>
> }
>
> }
>
> - if (pb[len] < cur[len])
>
> - {
>
> - *ptr1 = curMatch;
>
> - ptr1 = pair + 1;
>
> - curMatch = *ptr1;
>
> - len1 = len;
>
> - }
>
> - else
>
> {
>
> - *ptr0 = curMatch;
>
> - ptr0 = pair;
>
> - curMatch = *ptr0;
>
> - len0 = len;
>
> + UInt32 curMatch = pos - delta;
>
> + // delta = pos - *pair;
>
> + // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >>
31];
>
> + if (pb[len] < cur[len])
>
> + {
>
> + delta = pos - pair[1];
>
> + *ptr1 = curMatch;
>
> + ptr1 = pair + 1;
>
> + len1 = len;
>
> + }
>
> + else
>
> + {
>
> + delta = pos - *pair;
>
> + *ptr0 = curMatch;
>
> + ptr0 = pair;
>
> + len0 = len;
>
> + }
>
> }
>
> }
>
> + if (--cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + break;
>
> + }
>
> }
>
> pos++;
>
> _cyclicBufferPos++;
>
> cur++;
>
> {
>
> UInt32 num = (UInt32)(distances - _distances);
>
> - *_distances = num - 1;
>
> - _distances += num;
>
> - limit -= num;
>
> + _distances[-1] = num;
>
> }
>
> }
>
> - while (limit > 0 && --size != 0);
>
> + while (distances < limit && --size != 0);
>
> *posRes = pos;
>
> - return limit;
>
> + return distances;
>
> }
>
>
>
> #endif
>
>
>
> +
>
> +
>
> static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
>
> {
>
> UInt32 numProcessed = 0;
>
> UInt32 curPos = 2;
>
> - UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
>
> + UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
>
>
>
> distances[1] = p->hashNumAvail;
>
>
>
> @@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p,
> UInt32 *distances)
> #else
>
> {
>
> UInt32 posRes;
>
> - curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer,
p->son,
> cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> - distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
>
> + curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer,
> p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> + distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos,
>
> + distances + limit,
>
> + size, &posRes) - distances);
>
> p->hashBufPos += posRes - pos;
>
> cyclicBufferPos += posRes - pos;
>
> p->buffer += posRes - pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> index 3d86c788f3..fdd17008c2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> @@ -1,5 +1,5 @@
> /* LzFindMt.h -- multithreaded Match finder for LZ algorithms
>
> -2017-04-03 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __LZ_FIND_MT_H
>
> #define __LZ_FIND_MT_H
>
> @@ -19,10 +19,10 @@ EXTERN_C_BEGIN
>
>
> typedef struct _CMtSync
>
> {
>
> - Bool wasCreated;
>
> - Bool needStart;
>
> - Bool exit;
>
> - Bool stopWriting;
>
> + BoolInt wasCreated;
>
> + BoolInt needStart;
>
> + BoolInt exit;
>
> + BoolInt stopWriting;
>
>
>
> CThread thread;
>
> CAutoResetEvent canStart;
>
> @@ -30,8 +30,8 @@ typedef struct _CMtSync
> CAutoResetEvent wasStopped;
>
> CSemaphore freeSemaphore;
>
> CSemaphore filledSemaphore;
>
> - Bool csWasInitialized;
>
> - Bool csWasEntered;
>
> + BoolInt csWasInitialized;
>
> + BoolInt csWasEntered;
>
> CCriticalSection cs;
>
> UInt32 numProcessedBlocks;
>
> } CMtSync;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> index 962b94bb63..4d1576419f 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> @@ -1,13 +1,13 @@
> /* LzmaDec.c -- LZMA Decoder
>
> -2018-02-28 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> +#include <string.h>
>
> +
>
> /* #include "CpuArch.h" */
>
> #include "LzmaDec.h"
>
>
>
> -#include <string.h>
>
> -
>
> #define kNumTopBits 24
>
> #define kTopValue ((UInt32)1 << kNumTopBits)
>
>
>
> @@ -19,7 +19,7 @@
>
>
> #define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code <<
8)
> | (*buf++); }
>
>
>
> -#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt +
> ((kBitModelTotal - ttt) >> kNumMoveBits));
>
> #define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt
> - (ttt >> kNumMoveBits));
>
> #define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
>
> @@ -66,7 +66,7 @@
>
>
> #define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit)
> return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
>
>
>
> -#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0_CHECK range = bound;
>
> #define UPDATE_1_CHECK range -= bound; code -= bound;
>
> #define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
>
> @@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> curLen = ((rem < len) ? (unsigned)rem : len);
>
> pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
>
>
>
> - processedPos += curLen;
>
> + processedPos += (UInt32)curLen;
>
>
>
> len -= curLen;
>
> if (curLen <= dicBufSize - pos)
>
> @@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> Byte *dest = dic + dicPos;
>
> ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
>
> const Byte *lim = dest + curLen;
>
> - dicPos += curLen;
>
> + dicPos += (SizeT)curLen;
>
> do
>
> *(dest) = (Byte)*(dest + src);
>
> while (++dest != lim);
>
> @@ -572,14 +572,14 @@ int MY_FAST_CALL
> LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
> p->buf = buf;
>
> p->range = range;
>
> p->code = code;
>
> - p->remainLen = len;
>
> + p->remainLen = (UInt32)len;
>
> p->dicPos = dicPos;
>
> p->processedPos = processedPos;
>
> p->reps[0] = rep0;
>
> p->reps[1] = rep1;
>
> p->reps[2] = rep2;
>
> p->reps[3] = rep3;
>
> - p->state = state;
>
> + p->state = (UInt32)state;
>
>
>
> return SZ_OK;
>
> }
>
> @@ -601,8 +601,8 @@ static void MY_FAST_CALL
> LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
> if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
>
> p->checkDicSize = p->prop.dicSize;
>
>
>
> - p->processedPos += len;
>
> - p->remainLen -= len;
>
> + p->processedPos += (UInt32)len;
>
> + p->remainLen -= (UInt32)len;
>
> while (len != 0)
>
> {
>
> len--;
>
> @@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const
> CLzmaDec *p, const Byte *buf, SizeT inS
> }
>
>
>
>
>
> -void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
>
> +void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt
> initState)
>
> {
>
> p->remainLen = kMatchSpecLenStart + 1;
>
> p->tempBufSize = 0;
>
> @@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> p->tempBufSize = rem;
>
> if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
>
> {
>
> - int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
>
> + int dummyRes = LzmaDec_TryDummy(p, p->tempBuf,
> (SizeT)rem);
>
> if (dummyRes == DUMMY_ERROR)
>
> {
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> *status = LZMA_STATUS_NEEDS_MORE_INPUT;
>
> return SZ_OK;
>
> }
>
> @@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> return SZ_ERROR_FAIL; /* some internal error */
>
> lookAhead -= rem;
>
> }
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> src += lookAhead;
>
> - inSize -= lookAhead;
>
> + inSize -= (SizeT)lookAhead;
>
> p->tempBufSize = 0;
>
> }
>
> }
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> index e281716fee..4e9b499f8d 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> @@ -1,5 +1,5 @@
> /* LzmaEnc.c -- LZMA Encoder
>
> -2018-04-29 : Igor Pavlov : Public domain */
>
> +2019-01-10: Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
> #define kNumBitPriceShiftBits 4
>
> #define kBitPrice (1 << kNumBitPriceShiftBits)
>
>
>
> +#define REP_LEN_COUNT 64
>
> +
>
> void LzmaEncProps_Init(CLzmaEncProps *p)
>
> {
>
> p->level = 5;
>
> @@ -52,7 +54,7 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p)
> int level = p->level;
>
> if (level < 0) level = 5;
>
> p->level = level;
>
> -
>
> +
>
> if (p->dictSize == 0) p->dictSize = (level <= 5 ? (1 << (level * 2 +
14)) : (level
> <= 7 ? (1 << 25) : (1 << 26)));
>
> if (p->dictSize > p->reduceSize)
>
> {
>
> @@ -74,7 +76,7 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p)
> if (p->btMode < 0) p->btMode = (p->algo == 0 ? 0 : 1);
>
> if (p->numHashBytes < 0) p->numHashBytes = 4;
>
> if (p->mc == 0) p->mc = (16 + (p->fb >> 1)) >> (p->btMode ? 0 : 1);
>
> -
>
> +
>
> if (p->numThreads < 0)
>
> p->numThreads =
>
> #ifndef _7ZIP_ST
>
> @@ -124,7 +126,7 @@ static void LzmaEnc_FastPosInit(Byte *g_FastPos)
> g_FastPos[0] = 0;
>
> g_FastPos[1] = 1;
>
> g_FastPos += 2;
>
> -
>
> +
>
> for (slot = 2; slot < kNumLogBits * 2; slot++)
>
> {
>
> size_t k = ((size_t)1 << ((slot >> 1) - 1));
>
> @@ -183,8 +185,11 @@ typedef struct
> } COptimal;
>
>
>
>
>
> -#define kNumOpts (1 << 12)
>
> -#define kPackReserve (1 + kNumOpts * 2)
>
> +// 18.06
>
> +#define kNumOpts (1 << 11)
>
> +#define kPackReserve (kNumOpts * 8)
>
> +// #define kNumOpts (1 << 12)
>
> +// #define kPackReserve (1 + kNumOpts * 2)
>
>
>
> #define kNumLenToPosStates 4
>
> #define kNumPosSlotBits 6
>
> @@ -236,10 +241,18 @@ typedef struct
> typedef struct
>
> {
>
> unsigned tableSize;
>
> - unsigned counters[LZMA_NUM_PB_STATES_MAX];
>
> UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
>
> + // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols *
> 2];
>
> + // UInt32 prices2[kLenNumSymbolsTotal];
>
> } CLenPriceEnc;
>
>
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
>
> +
>
> +/*
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) -
2) &
> (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >>
9)))
>
> +*/
>
>
>
> typedef struct
>
> {
>
> @@ -273,7 +286,7 @@ typedef struct
>
>
> CLzmaProb posSlotEncoder[kNumLenToPosStates][1 <<
> kNumPosSlotBits];
>
> CLzmaProb posEncoders[kNumFullDistances];
>
> -
>
> +
>
> CLenEnc lenProbs;
>
> CLenEnc repLenProbs;
>
>
>
> @@ -308,16 +321,18 @@ typedef struct
> unsigned lc, lp, pb;
>
> unsigned lclp;
>
>
>
> - Bool fastMode;
>
> - Bool writeEndMark;
>
> - Bool finished;
>
> - Bool multiThread;
>
> - Bool needInit;
>
> + BoolInt fastMode;
>
> + BoolInt writeEndMark;
>
> + BoolInt finished;
>
> + BoolInt multiThread;
>
> + BoolInt needInit;
>
> + // BoolInt _maxMode;
>
>
>
> UInt64 nowPos64;
>
> -
>
> +
>
> unsigned matchPriceCount;
>
> - unsigned alignPriceCount;
>
> + // unsigned alignPriceCount;
>
> + int repLenEncCounter;
>
>
>
> unsigned distTableSize;
>
>
>
> @@ -325,7 +340,7 @@ typedef struct
> SRes result;
>
>
>
> #ifndef _7ZIP_ST
>
> - Bool mtMode;
>
> + BoolInt mtMode;
>
> // begin of CMatchFinderMt is used in LZ thread
>
> CMatchFinderMt matchFinderMt;
>
> // end of CMatchFinderMt is used in BT and HASH threads
>
> @@ -336,7 +351,7 @@ typedef struct
> #ifndef _7ZIP_ST
>
> Byte pad[128];
>
> #endif
>
> -
>
> +
>
> // LZ thread
>
> CProbPrice ProbPrices[kBitModelTotal >> kNumMoveReducingBits];
>
>
>
> @@ -355,7 +370,7 @@ typedef struct
> CLzmaProb isRep0Long[kNumStates][LZMA_NUM_PB_STATES_MAX];
>
> CLzmaProb posSlotEncoder[kNumLenToPosStates][1 <<
> kNumPosSlotBits];
>
> CLzmaProb posEncoders[kNumFullDistances];
>
> -
>
> +
>
> CLenEnc lenProbs;
>
> CLenEnc repLenProbs;
>
>
>
> @@ -383,9 +398,9 @@ void LzmaEnc_SaveState(CLzmaEncHandle pp)
> {
>
> CLzmaEnc *p = (CLzmaEnc *)pp;
>
> CSaveState *dest = &p->saveState;
>
> -
>
> +
>
> dest->state = p->state;
>
> -
>
> +
>
> dest->lenProbs = p->lenProbs;
>
> dest->repLenProbs = p->repLenProbs;
>
>
>
> @@ -414,9 +429,9 @@ void LzmaEnc_RestoreState(CLzmaEncHandle pp)
>
>
> dest->lenProbs = p->lenProbs;
>
> dest->repLenProbs = p->repLenProbs;
>
> -
>
> +
>
> COPY_ARR(dest, p, reps);
>
> -
>
> +
>
> COPY_ARR(dest, p, posAlignEncoder);
>
> COPY_ARR(dest, p, isRep);
>
> COPY_ARR(dest, p, isRepG0);
>
> @@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const
> CLzmaEncProps *props2)
> p->lp = props.lp;
>
> p->pb = props.pb;
>
> p->fastMode = (props.algo == 0);
>
> + // p->_maxMode = True;
>
> p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
>
> {
>
> unsigned numHashBytes = 4;
>
> @@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #ifdef _LZMA_ENC_USE_BRANCH
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> RC_BIT_PRE(p, prob) \
>
> - if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> + if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> else { (p)->low += newBound; range -= newBound; ttt -= ttt >>
> kNumMoveBits; } \
>
> *(prob) = (CLzmaProb)ttt; \
>
> RC_NORM(p) \
>
> @@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #else
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> UInt32 mask; \
>
> RC_BIT_PRE(p, prob) \
>
> - mask = 0 - (UInt32)symbol; \
>
> + mask = 0 - (UInt32)bit; \
>
> range &= mask; \
>
> mask &= newBound; \
>
> range -= mask; \
>
> (p)->low += mask; \
>
> - mask = (UInt32)symbol - 1; \
>
> + mask = (UInt32)bit - 1; \
>
> range += newBound & mask; \
>
> mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
>
> mask += ((1 << kNumMoveBits) - 1); \
>
> @@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p,
> CLzmaProb *prob)
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
>
> +static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
>
> {
>
> UInt32 range = p->range;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
>
> - CLzmaProb *prob = probs + (symbol >> 8);
>
> - UInt32 bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> + // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
>
> + CLzmaProb *prob = probs + (sym >> 8);
>
> + UInt32 bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte)
>
> +static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs,
> UInt32 sym, UInt32 matchByte)
>
> {
>
> UInt32 range = p->range;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> CLzmaProb *prob;
>
> UInt32 bit;
>
> matchByte <<= 1;
>
> - // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol
>>
> 8)), (symbol >> 7) & 1);
>
> - prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
>
> - bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >>
> 8)), (sym >> 7) & 1);
>
> + prob = probs + (offs + (matchByte & offs) + (sym >> 8));
>
> + bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> @@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> }
>
>
>
>
>
> -#define GET_PRICE(prob, symbol) \
>
> - p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICE(prob, bit) \
>
> + p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> -#define GET_PRICEa(prob, symbol) \
>
> - ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICEa(prob, bit) \
>
> + ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> #define GET_PRICE_0(prob) p->ProbPrices[(prob) >>
> kNumMoveReducingBits]
>
> #define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1))
>>
> kNumMoveReducingBits]
>
> @@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> #define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >>
> kNumMoveReducingBits]
>
>
>
>
>
> -static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol,
const
> CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const
> CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> - price += GET_PRICEa(probs[symbol], bit);
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> }
>
> - while (symbol >= 2);
>
> + while (sym >= 2);
>
> return price;
>
> }
>
>
>
>
>
> -static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> sym, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> matchByte <<= 1;
>
> - price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)],
> (symbol >> 7) & 1);
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)],
> (sym >> 7) & 1);
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> return price;
>
> }
>
>
>
>
>
> -static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, UInt32 symbol)
>
> +static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, unsigned sym)
>
> {
>
> UInt32 range = rc->range;
>
> unsigned m = 1;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - unsigned bit = symbol & 1;
>
> + unsigned bit = sym & 1;
>
> // RangeEnc_EncodeBit(rc, probs + m, bit);
>
> - symbol >>= 1;
>
> + sym >>= 1;
>
> RC_BIT(rc, probs + m, bit);
>
> m = (m << 1) | bit;
>
> }
>
> @@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
> p->high[i] = kProbInitValue;
>
> }
>
>
>
> -static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol,
> unsigned posState)
>
> +static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym,
> unsigned posState)
>
> {
>
> UInt32 range, ttt, newBound;
>
> CLzmaProb *probs = p->low;
>
> range = rc->range;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols)
>
> + if (sym >= kLenNumLowSymbols)
>
> {
>
> RC_BIT_1(rc, probs);
>
> probs += kLenNumLowSymbols;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols * 2)
>
> + if (sym >= kLenNumLowSymbols * 2)
>
> {
>
> RC_BIT_1(rc, probs);
>
> rc->range = range;
>
> - // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol -
> kLenNumLowSymbols * 2);
>
> - LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
>
> + // RcTree_Encode(rc, p->high, kLenNumHighBits, sym -
> kLenNumLowSymbols * 2);
>
> + LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
>
> return;
>
> }
>
> - symbol -= kLenNumLowSymbols;
>
> + sym -= kLenNumLowSymbols;
>
> }
>
>
>
> - // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, symbol);
>
> + // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, sym);
>
> {
>
> unsigned m;
>
> unsigned bit;
>
> RC_BIT_0(rc, probs);
>
> probs += (posState << (1 + kLenNumLowBits));
>
> - bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) +
bit;
>
> - bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) +
bit;
>
> - bit = symbol & 1; RC_BIT(rc, probs + m, bit);
>
> + bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
>
> + bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
>
> + bit = sym & 1; RC_BIT(rc, probs + m, bit);
>
> rc->range = range;
>
> }
>
> }
>
> @@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs,
> UInt32 startPrice, UInt32 *price
> }
>
>
>
>
>
> -MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
>
> - CLenPriceEnc *p, unsigned posState,
>
> +MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
>
> + CLenPriceEnc *p,
>
> + unsigned numPosStates,
>
> const CLenEnc *enc,
>
> const CProbPrice *ProbPrices)
>
> {
>
> - // int y; for (y = 0; y < 100; y++) {
>
> - UInt32 a;
>
> - unsigned i, numSymbols;
>
> + UInt32 b;
>
> +
>
> + {
>
> + unsigned prob = enc->low[0];
>
> + UInt32 a, c;
>
> + unsigned posState;
>
> + b = GET_PRICEa_1(prob);
>
> + a = GET_PRICEa_0(prob);
>
> + c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (posState = 0; posState < numPosStates; posState++)
>
> + {
>
> + UInt32 *prices = p->prices[posState];
>
> + const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> + SetPrices_3(probs, a, prices, ProbPrices);
>
> + SetPrices_3(probs + kLenNumLowSymbols, c, prices +
> kLenNumLowSymbols, ProbPrices);
>
> + }
>
> + }
>
>
>
> - UInt32 *prices = p->prices[posState];
>
> + /*
>
> {
>
> - const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> - SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
>
> + unsigned i;
>
> + UInt32 b;
>
> + a = GET_PRICEa_0(enc->low[0]);
>
> + for (i = 0; i < kLenNumLowSymbols; i++)
>
> + p->prices2[i] = a;
>
> a = GET_PRICEa_1(enc->low[0]);
>
> - SetPrices_3(probs + kLenNumLowSymbols, a +
> GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices +
> kLenNumLowSymbols, ProbPrices);
>
> + b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
>
> + p->prices2[i] = b;
>
> a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> }
>
> - numSymbols = p->tableSize;
>
> - p->counters[posState] = numSymbols;
>
> - for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
>
> + */
>
> +
>
> + // p->counter = numSymbols;
>
> + // p->counter = 64;
>
> +
>
> {
>
> - prices[i] = a +
>
> - // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> - LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2,
> ProbPrices);
>
> - /*
>
> - unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
>
> - UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> - UInt32 prob = enc->high[(1 << 7) + sym];
>
> - prices[i ] = price + GET_PRICEa_0(prob);
>
> - prices[i + 1] = price + GET_PRICEa_1(prob);
>
> - */
>
> - }
>
> - // }
>
> -}
>
> + unsigned i = p->tableSize;
>
>
>
> -static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned
> numPosStates,
>
> - const CLenEnc *enc,
>
> - const CProbPrice *ProbPrices)
>
> -{
>
> - unsigned posState;
>
> - for (posState = 0; posState < numPosStates; posState++)
>
> - LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
>
> -}
>
> + if (i > kLenNumLowSymbols * 2)
>
> + {
>
> + const CLzmaProb *probs = enc->high;
>
> + UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
>
> + i -= kLenNumLowSymbols * 2 - 1;
>
> + i >>= 1;
>
> + b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> + do
>
> + {
>
> + /*
>
> + p->prices2[i] = a +
>
> + // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> + LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
>
> + */
>
> + // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> + unsigned sym = --i + (1 << (kLenNumHighBits - 1));
>
> + UInt32 price = b;
>
> + do
>
> + {
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> + }
>
> + while (sym >= 2);
>
> +
>
> + {
>
> + unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits -
1))];
>
> + prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
>
> + prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
>
> + }
>
> + }
>
> + while (i);
>
>
>
> + {
>
> + unsigned posState;
>
> + size_t num = (p->tableSize - kLenNumLowSymbols * 2) *
> sizeof(p->prices[0][0]);
>
> + for (posState = 1; posState < numPosStates; posState++)
>
> + memcpy(p->prices[posState] + kLenNumLowSymbols * 2,
> p->prices[0] + kLenNumLowSymbols * 2, num);
>
> + }
>
> + }
>
> + }
>
> +}
>
>
>
> /*
>
> #ifdef SHOW_STAT
>
> @@ -914,21 +973,21 @@ static void
> LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
> printf("\n MovePos %u", num);
>
> #endif
>
> */
>
> -
>
> +
>
> #define MOVE_POS(p, num) { \
>
> p->additionalOffset += (num); \
>
> - p->matchFinder.Skip(p->matchFinderObj, (num)); }
>
> + p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
>
>
>
>
>
> static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
>
> {
>
> unsigned numPairs;
>
> -
>
> +
>
> p->additionalOffset++;
>
> p->numAvail =
> p->matchFinder.GetNumAvailableBytes(p->matchFinderObj);
>
> numPairs = p->matchFinder.GetMatches(p->matchFinderObj,
> p->matches);
>
> *numPairsRes = numPairs;
>
> -
>
> +
>
> #ifdef SHOW_STAT
>
> printf("\n i = %u numPairs = %u ", g_STAT_OFFSET, numPairs / 2);
>
> g_STAT_OFFSET++;
>
> @@ -938,7 +997,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> printf("%2u %6u | ", p->matches[i], p->matches[i + 1]);
>
> }
>
> #endif
>
> -
>
> +
>
> if (numPairs == 0)
>
> return 0;
>
> {
>
> @@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> const Byte *p2 = p1 + len;
>
> ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
>
> const Byte *lim = p1 + numAvail;
>
> - for (; p2 != lim && *p2 == p2[dif]; p2++);
>
> + for (; p2 != lim && *p2 == p2[dif]; p2++)
>
> + {}
>
> return (unsigned)(p2 - p1);
>
> }
>
> }
>
> @@ -976,8 +1036,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> + GET_PRICE_1(p->isRep0Long[state][posState])) \
>
> + GET_PRICE_1(p->isRep[state]) \
>
> + GET_PRICE_0(p->isRepG0[state])
>
> -
>
>
>
> +MY_FORCE_INLINE
>
> static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex,
> size_t state, size_t posState)
>
> {
>
> UInt32 price;
>
> @@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p,
> unsigned cur)
> for (;;)
>
> {
>
> UInt32 dist = p->opt[cur].dist;
>
> - UInt32 len = p->opt[cur].len;
>
> - UInt32 extra = p->opt[cur].extra;
>
> + unsigned len = (unsigned)p->opt[cur].len;
>
> + unsigned extra = (unsigned)p->opt[cur].extra;
>
> cur -= len;
>
>
>
> if (extra)
>
> {
>
> wr--;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> cur -= extra;
>
> len = extra;
>
> if (extra == 1)
>
> @@ -1042,10 +1102,10 @@ static unsigned Backward(CLzmaEnc *p,
> unsigned cur)
> p->optCur = wr;
>
> return len;
>
> }
>
> -
>
> +
>
> wr--;
>
> p->opt[wr].dist = dist;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> }
>
> }
>
>
>
> @@ -1068,9 +1128,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> UInt32 matchPrice, repMatchPrice;
>
> const Byte *data;
>
> Byte curByte, matchByte;
>
> -
>
> +
>
> p->optCur = p->optEnd = 0;
>
> -
>
> +
>
> if (p->additionalOffset == 0)
>
> mainLen = ReadMatchDistances(p, &numPairs);
>
> else
>
> @@ -1078,7 +1138,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> mainLen = p->longestMatchLen;
>
> numPairs = p->numPairs;
>
> }
>
> -
>
> +
>
> numAvail = p->numAvail;
>
> if (numAvail < 2)
>
> {
>
> @@ -1087,10 +1147,10 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> if (numAvail > LZMA_MATCH_LEN_MAX)
>
> numAvail = LZMA_MATCH_LEN_MAX;
>
> -
>
> +
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> repMaxIndex = 0;
>
> -
>
> +
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> {
>
> unsigned len;
>
> @@ -1102,43 +1162,48 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> repLens[i] = 0;
>
> continue;
>
> }
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> repLens[i] = len;
>
> if (len > repLens[repMaxIndex])
>
> repMaxIndex = i;
>
> }
>
> -
>
> +
>
> if (repLens[repMaxIndex] >= p->numFastBytes)
>
> {
>
> unsigned len;
>
> - p->backRes = repMaxIndex;
>
> + p->backRes = (UInt32)repMaxIndex;
>
> len = repLens[repMaxIndex];
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> }
>
> -
>
> +
>
> matches = p->matches;
>
> -
>
> +
>
> if (mainLen >= p->numFastBytes)
>
> {
>
> p->backRes = matches[(size_t)numPairs - 1] + LZMA_NUM_REPS;
>
> MOVE_POS(p, mainLen - 1)
>
> return mainLen;
>
> }
>
> -
>
> +
>
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
> -
>
> - if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] <
> 2)
>
> +
>
> + last = repLens[repMaxIndex];
>
> + if (last <= mainLen)
>
> + last = mainLen;
>
> +
>
> + if (last < 2 && curByte != matchByte)
>
> {
>
> p->backRes = MARK_LIT;
>
> return 1;
>
> }
>
> -
>
> +
>
> p->opt[0].state = (CState)p->state;
>
> -
>
> +
>
> posState = (position & p->pbMask);
>
> -
>
> +
>
> {
>
> const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
>
> p->opt[1].price = GET_PRICE_0(p->isMatch[p->state][posState]) +
>
> @@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> LitEnc_Matched_GetPrice(probs, curByte, matchByte,
> p->ProbPrices) :
>
> LitEnc_GetPrice(probs, curByte, p->ProbPrices));
>
> }
>
> -
>
> +
>
> MakeAs_Lit(&p->opt[1]);
>
> -
>
> +
>
> matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
>
> -
>
> - if (matchByte == curByte)
>
> +
>
> + // 18.06
>
> + if (matchByte == curByte && repLens[0] == 0)
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p,
> p->state, posState);
>
> if (shortRepPrice < p->opt[1].price)
>
> @@ -1160,32 +1226,22 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[1].price = shortRepPrice;
>
> MakeAs_ShortRep(&p->opt[1]);
>
> }
>
> + if (last < 2)
>
> + {
>
> + p->backRes = p->opt[1].dist;
>
> + return 1;
>
> + }
>
> }
>
> -
>
> - last = (mainLen >= repLens[repMaxIndex] ? mainLen :
> repLens[repMaxIndex]);
>
> -
>
> - if (last < 2)
>
> - {
>
> - p->backRes = p->opt[1].dist;
>
> - return 1;
>
> - }
>
> -
>
> +
>
> p->opt[1].len = 1;
>
> -
>
> +
>
> p->opt[0].reps[0] = reps[0];
>
> p->opt[0].reps[1] = reps[1];
>
> p->opt[0].reps[2] = reps[2];
>
> p->opt[0].reps[3] = reps[3];
>
> -
>
> - {
>
> - unsigned len = last;
>
> - do
>
> - p->opt[len--].price = kInfinityPrice;
>
> - while (len >= 2);
>
> - }
>
>
>
> // ---------- REP ----------
>
> -
>
> +
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> {
>
> unsigned repLen = repLens[i];
>
> @@ -1195,58 +1251,61 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)repLen - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState,
> repLen);
>
> COptimal *opt = &p->opt[repLen];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = repLen;
>
> - opt->dist = i;
>
> + opt->len = (UInt32)repLen;
>
> + opt->dist = (UInt32)i;
>
> opt->extra = 0;
>
> }
>
> }
>
> while (--repLen >= 2);
>
> }
>
> -
>
> -
>
> +
>
> +
>
> // ---------- MATCH ----------
>
> {
>
> - unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
>
> + unsigned len = repLens[0] + 1;
>
> if (len <= mainLen)
>
> {
>
> unsigned offs = 0;
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[p->state]);
>
>
>
> - while (len > matches[offs])
>
> - offs += 2;
>
> -
>
> + if (len < 2)
>
> + len = 2;
>
> + else
>
> + while (len > matches[offs])
>
> + offs += 2;
>
> +
>
> for (; ; len++)
>
> {
>
> COptimal *opt;
>
> UInt32 dist = matches[(size_t)offs + 1];
>
> - UInt32 price2 = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice +
> GET_PRICE_LEN(&p->lenEnc, posState, len);
>
> unsigned lenToPosState = GetLenToPosState(len);
>
> -
>
> +
>
> if (dist < kNumFullDistances)
>
> - price2 += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> else
>
> {
>
> unsigned slot;
>
> GetPosSlot2(dist, slot);
>
> - price2 += p->alignPrices[dist & kAlignMask];
>
> - price2 += p->posSlotPrices[lenToPosState][slot];
>
> + price += p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenToPosState][slot];
>
> }
>
> -
>
> +
>
> opt = &p->opt[len];
>
> -
>
> - if (price2 < opt->price)
>
> +
>
> + if (price < opt->price)
>
> {
>
> - opt->price = price2;
>
> - opt->len = len;
>
> + opt->price = price;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> -
>
> +
>
> if (len == matches[offs])
>
> {
>
> offs += 2;
>
> @@ -1256,7 +1315,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> }
>
> }
>
> }
>
> -
>
> +
>
>
>
> cur = 0;
>
>
>
> @@ -1272,37 +1331,73 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
>
>
>
>
> -
>
> +
>
> // ---------- Optimal Parsing ----------
>
>
>
> for (;;)
>
> {
>
> - UInt32 numAvail, numAvailFull;
>
> + unsigned numAvail;
>
> + UInt32 numAvailFull;
>
> unsigned newLen, numPairs, prev, state, posState, startLen;
>
> - UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
>
> - Bool nextIsLit;
>
> + UInt32 litPrice, matchPrice, repMatchPrice;
>
> + BoolInt nextIsLit;
>
> Byte curByte, matchByte;
>
> const Byte *data;
>
> COptimal *curOpt, *nextOpt;
>
>
>
> if (++cur == last)
>
> - return Backward(p, cur);
>
> + break;
>
> +
>
> + // 18.06
>
> + if (cur >= kNumOpts - 64)
>
> + {
>
> + unsigned j, best;
>
> + UInt32 price = p->opt[cur].price;
>
> + best = cur;
>
> + for (j = cur + 1; j <= last; j++)
>
> + {
>
> + UInt32 price2 = p->opt[j].price;
>
> + if (price >= price2)
>
> + {
>
> + price = price2;
>
> + best = j;
>
> + }
>
> + }
>
> + {
>
> + unsigned delta = best - cur;
>
> + if (delta != 0)
>
> + {
>
> + MOVE_POS(p, delta);
>
> + }
>
> + }
>
> + cur = best;
>
> + break;
>
> + }
>
>
>
> newLen = ReadMatchDistances(p, &numPairs);
>
> -
>
> +
>
> if (newLen >= p->numFastBytes)
>
> {
>
> p->numPairs = numPairs;
>
> p->longestMatchLen = newLen;
>
> - return Backward(p, cur);
>
> + break;
>
> }
>
> -
>
> +
>
> curOpt = &p->opt[cur];
>
> +
>
> + position++;
>
> +
>
> + // we need that check here, if skip_items in p->opt are possible
>
> + /*
>
> + if (curOpt->price >= kInfinityPrice)
>
> + continue;
>
> + */
>
> +
>
> prev = cur - curOpt->len;
>
> -
>
> +
>
> if (curOpt->len == 1)
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (IsShortRep(curOpt))
>
> state = kShortRepNextStates[state];
>
> else
>
> @@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> if (curOpt->extra)
>
> {
>
> - prev -= curOpt->extra;
>
> + prev -= (unsigned)curOpt->extra;
>
> state = kState_RepAfterLit;
>
> if (curOpt->extra == 1)
>
> - state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit :
> kState_MatchAfterLit;
>
> + state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit :
> kState_MatchAfterLit);
>
> }
>
> else
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (dist < LZMA_NUM_REPS)
>
> state = kRepNextStates[state];
>
> else
>
> @@ -1368,7 +1463,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> reps[3] = prevOpt->reps[2];
>
> }
>
> }
>
> -
>
> +
>
> curOpt->state = (CState)state;
>
> curOpt->reps[0] = reps[0];
>
> curOpt->reps[1] = reps[1];
>
> @@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
>
>
> - position++;
>
> posState = (position & p->pbMask);
>
>
>
> /*
>
> @@ -1391,19 +1485,31 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> < MATCH [ : LIT : REP_0 ]
>
> */
>
>
>
> - curPrice = curOpt->price;
>
> - litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
>
> + {
>
> + UInt32 curPrice = curOpt->price;
>
> + unsigned prob = p->isMatch[state][posState];
>
> + matchPrice = curPrice + GET_PRICE_1(prob);
>
> + litPrice = curPrice + GET_PRICE_0(prob);
>
> + }
>
>
>
> nextOpt = &p->opt[(size_t)cur + 1];
>
> nextIsLit = False;
>
>
>
> - // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
>
> + // here we can allow skip_items in p->opt, if we don't check
> (nextOpt->price < kInfinityPrice)
>
> + // 18.new.06
>
> + if ((nextOpt->price < kInfinityPrice
>
> + // && !IsLitState(state)
>
> + && matchByte == curByte)
>
> + || litPrice > nextOpt->price
>
> + )
>
> + litPrice = 0;
>
> + else
>
> {
>
> const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
>
> litPrice += (!IsLitState(state) ?
>
> LitEnc_Matched_GetPrice(probs, curByte, matchByte,
> p->ProbPrices) :
>
> LitEnc_GetPrice(probs, curByte, p->ProbPrices));
>
> -
>
> +
>
> if (litPrice < nextOpt->price)
>
> {
>
> nextOpt->price = litPrice;
>
> @@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
>
> -
>
> +
>
> + numAvailFull = p->numAvail;
>
> + {
>
> + unsigned temp = kNumOpts - 1 - cur;
>
> + if (numAvailFull > temp)
>
> + numAvailFull = (UInt32)temp;
>
> + }
>
> +
>
> + // 18.06
>
> // ---------- SHORT_REP ----------
>
> - // if (IsLitState(state)) // 18.new
>
> + if (IsLitState(state)) // 18.new
>
> if (matchByte == curByte)
>
> - // if (repMatchPrice < nextOpt->price) // 18.new
>
> - if (nextOpt->len < 2
>
> + if (repMatchPrice < nextOpt->price) // 18.new
>
> + // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
>
> + if (
>
> + // nextOpt->price >= kInfinityPrice ||
>
> + nextOpt->len < 2 // we can check nextOpt->len, if skip items
> are not allowed in p->opt
>
> || (nextOpt->dist != 0
>
> - && nextOpt->extra <= 1 // 17.old
>
> - ))
>
> + // && nextOpt->extra <= 1 // 17.old
>
> + )
>
> + )
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state,
> posState);
>
> - if (shortRepPrice <= nextOpt->price) // 17.old
>
> - // if (shortRepPrice < nextOpt->price) // 18.new
>
> + // if (shortRepPrice <= nextOpt->price) // 17.old
>
> + if (shortRepPrice < nextOpt->price) // 18.new
>
> {
>
> nextOpt->price = shortRepPrice;
>
> nextOpt->len = 1;
>
> @@ -1435,13 +1552,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> nextIsLit = False;
>
> }
>
> }
>
> -
>
> - numAvailFull = p->numAvail;
>
> - {
>
> - UInt32 temp = kNumOpts - 1 - cur;
>
> - if (numAvailFull > temp)
>
> - numAvailFull = temp;
>
> - }
>
>
>
> if (numAvailFull < 2)
>
> continue;
>
> @@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
>
>
> // ---------- LIT : REP_0 ----------
>
>
>
> - if (
>
> - // litPrice != 0 && // 18.new
>
> - !nextIsLit
>
> + if (!nextIsLit
>
> + && litPrice != 0 // 18.new
>
> && matchByte != curByte
>
> && numAvailFull > 2)
>
> {
>
> @@ -1465,32 +1574,32 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> for (len = 3; len < limit && data[len] == data2[len]; len++)
>
> - {
>
> - }
>
> -
>
> + {}
>
> +
>
> {
>
> unsigned state2 = kLiteralNextStates[state];
>
> unsigned posState2 = (position + 1) & p->pbMask;
>
> UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
>
> {
>
> unsigned offset = cur + len;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> -
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> +
>
> // do
>
> {
>
> UInt32 price2;
>
> COptimal *opt;
>
> len--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = 0;
>
> opt->extra = 1;
>
> }
>
> @@ -1500,8 +1609,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> }
>
> }
>
> }
>
> -
>
> +
>
> startLen = 2; /* speed optimization */
>
> +
>
> {
>
> // ---------- REP ----------
>
> unsigned repIndex = 0; // 17.old
>
> @@ -1513,31 +1623,35 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> const Byte *data2 = data - reps[repIndex];
>
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
> -
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> -
>
> +
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> +
>
> // if (len < startLen) continue; // 18.new: speed optimization
>
>
>
> - while (last < cur + len)
>
> - p->opt[++last].price = kInfinityPrice;
>
> + {
>
> + unsigned offset = cur + len;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
> {
>
> unsigned len2 = len;
>
> price = repMatchPrice + GetPrice_PureRep(p, repIndex, state,
> posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)len2 - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState, len2);
>
> COptimal *opt = &p->opt[cur + len2];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> - opt->dist = repIndex;
>
> + opt->len = (UInt32)len2;
>
> + opt->dist = (UInt32)repIndex;
>
> opt->extra = 0;
>
> }
>
> }
>
> while (--len2 >= 2);
>
> }
>
> -
>
> +
>
> if (repIndex == 0) startLen = len + 1; // 17.old
>
> // startLen = len + 1; // 18.new
>
>
>
> @@ -1550,51 +1664,59 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> unsigned limit = len2 + p->numFastBytes;
>
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> -
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> -
>
> - len2 -= len;
>
> - if (len2 >= 3)
>
> +
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> {
>
> unsigned state2 = kRepNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> - price +=
>
> - p->repLenEnc.prices[posState][(size_t)len - 2]
>
> + price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
>
> + GET_PRICE_0(p->isMatch[state2][posState2])
>
> + LitEnc_Matched_GetPrice(LIT_PROBS(position + len,
> data[(size_t)len - 1]),
>
> data[len], data2[len], p->ProbPrices);
>
> -
>
> +
>
> // state2 = kLiteralNextStates[state2];
>
> state2 = kState_LitAfterRep;
>
> posState2 = (posState2 + 1) & p->pbMask;
>
>
>
>
>
> price += GetPrice_Rep_0(p, state2, posState2);
>
> +
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
> +
>
> + len2 -= len;
>
> + // if (len2 >= 3)
>
> + {
>
> {
>
> unsigned offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> - unsigned price2;
>
> + UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> - opt->dist = repIndex;
>
> + opt->dist = (UInt32)repIndex;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> }
>
> + }
>
> }
>
> }
>
> }
>
> @@ -1606,49 +1728,57 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> newLen = numAvail;
>
> for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
>
> - matches[numPairs] = newLen;
>
> + matches[numPairs] = (UInt32)newLen;
>
> numPairs += 2;
>
> }
>
> -
>
> +
>
> + // startLen = 2; /* speed optimization */
>
> +
>
> if (newLen >= startLen)
>
> {
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[state]);
>
> UInt32 dist;
>
> unsigned offs, posSlot, len;
>
> - while (last < cur + newLen)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + {
>
> + unsigned offset = cur + newLen;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
>
>
> offs = 0;
>
> while (startLen > matches[offs])
>
> offs += 2;
>
> dist = matches[(size_t)offs + 1];
>
> -
>
> +
>
> // if (dist >= kNumFullDistances)
>
> GetPosSlot2(dist, posSlot);
>
> -
>
> +
>
> for (len = /*2*/ startLen; ; len++)
>
> {
>
> - UInt32 price = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc,
> posState, len);
>
> {
>
> COptimal *opt;
>
> - unsigned lenToPosState = len - 2; lenToPosState =
> GetLenToPosState2(lenToPosState);
>
> + unsigned lenNorm = len - 2;
>
> + lenNorm = GetLenToPosState2(lenNorm);
>
> if (dist < kNumFullDistances)
>
> - price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenNorm][dist &
> (kNumFullDistances - 1)];
>
> else
>
> - price += p->posSlotPrices[lenToPosState][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
> -
>
> + price += p->posSlotPrices[lenNorm][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
> +
>
> opt = &p->opt[cur + len];
>
> if (price < opt->price)
>
> {
>
> opt->price = price;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> }
>
>
>
> - if (/*_maxMode && */ len == matches[offs])
>
> + if (len == matches[offs])
>
> {
>
> + // if (p->_maxMode) {
>
> // MATCH : LIT : REP_0
>
>
>
> const Byte *data2 = data - dist - 1;
>
> @@ -1656,12 +1786,18 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> unsigned limit = len2 + p->numFastBytes;
>
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> -
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> -
>
> +
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> + {
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
> +
>
> len2 -= len;
>
> -
>
> - if (len2 >= 3)
>
> +
>
> + // if (len2 >= 3)
>
> {
>
> unsigned state2 = kMatchNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> @@ -1677,28 +1813,31 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price += GetPrice_Rep_0(p, state2, posState2);
>
>
>
> offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> -
>
> +
>
> + }
>
> +
>
> offs += 2;
>
> if (offs == numPairs)
>
> break;
>
> @@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
> }
>
> +
>
> + do
>
> + p->opt[last].price = kInfinityPrice;
>
> + while (--last);
>
> +
>
> + return Backward(p, cur);
>
> }
>
>
>
>
>
> @@ -1735,21 +1880,23 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> p->backRes = MARK_LIT;
>
> if (numAvail < 2)
>
> return 1;
>
> + // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
>
> if (numAvail > LZMA_MATCH_LEN_MAX)
>
> numAvail = LZMA_MATCH_LEN_MAX;
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> repLen = repIndex = 0;
>
> -
>
> +
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> {
>
> unsigned len;
>
> const Byte *data2 = data - p->reps[i];
>
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> if (len >= p->numFastBytes)
>
> {
>
> - p->backRes = i;
>
> + p->backRes = (UInt32)i;
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> }
>
> @@ -1768,7 +1915,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> }
>
>
>
> mainDist = 0; /* for GCC */
>
> -
>
> +
>
> if (mainLen >= 2)
>
> {
>
> mainDist = p->matches[(size_t)numPairs - 1];
>
> @@ -1793,18 +1940,18 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> || (repLen + 2 >= mainLen && mainDist >= (1 << 9))
>
> || (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
>
> {
>
> - p->backRes = repIndex;
>
> + p->backRes = (UInt32)repIndex;
>
> MOVE_POS(p, repLen - 1)
>
> return repLen;
>
> }
>
> -
>
> +
>
> if (mainLen < 2 || numAvail <= 2)
>
> return 1;
>
>
>
> {
>
> unsigned len1 = ReadMatchDistances(p, &p->numPairs);
>
> p->longestMatchLen = len1;
>
> -
>
> +
>
> if (len1 >= 2)
>
> {
>
> UInt32 newDist = p->matches[(size_t)p->numPairs - 1];
>
> @@ -1815,9 +1962,9 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> return 1;
>
> }
>
> }
>
> -
>
> +
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> -
>
> +
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> {
>
> unsigned len, limit;
>
> @@ -1833,7 +1980,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> break;
>
> }
>
> }
>
> -
>
> +
>
> p->backRes = mainDist + LZMA_NUM_REPS;
>
> if (mainLen != 2)
>
> {
>
> @@ -1859,7 +2006,7 @@ static void WriteEndMarker(CLzmaEnc *p,
> unsigned posState)
> RC_BIT_0(&p->rc, prob)
>
> }
>
> p->state = kMatchNextStates[p->state];
>
> -
>
> +
>
> p->rc.range = range;
>
> LenEnc_Encode(&p->lenProbs, &p->rc, 0, posState);
>
> range = p->rc.range;
>
> @@ -1888,7 +2035,7 @@ static void WriteEndMarker(CLzmaEnc *p,
> unsigned posState)
> }
>
> while (--numBits);
>
> }
>
> -
>
> +
>
> {
>
> // RcTree_ReverseEncode(&p->rc, p->posAlignEncoder, kNumAlignBits,
> kAlignMask);
>
> CLzmaProb *probs = p->posAlignEncoder;
>
> @@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p,
> UInt32 nowPos)
> }
>
>
>
>
>
> -
>
> -static void FillAlignPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
>
> {
>
> unsigned i;
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> const CLzmaProb *probs = p->posAlignEncoder;
>
> - p->alignPriceCount = 0;
>
> + // p->alignPriceCount = 0;
>
> for (i = 0; i < kAlignTableSize / 2; i++)
>
> {
>
> UInt32 price = 0;
>
> - unsigned symbol = i;
>
> + unsigned sym = i;
>
> unsigned m = 1;
>
> unsigned bit;
>
> UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> prob = probs[m];
>
> p->alignPrices[i ] = price + GET_PRICEa_0(prob);
>
> p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
>
> @@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
> }
>
>
>
>
>
> -static void FillDistancesPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
>
> {
>
> + // int y; for (y = 0; y < 100; y++) {
>
> +
>
> UInt32 tempPrices[kNumFullDistances];
>
> - unsigned i, lenToPosState;
>
> + unsigned i, lps;
>
>
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> p->matchPriceCount = 0;
>
>
>
> - for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
>
> + for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
>
> {
>
> unsigned posSlot = GetPosSlot1(i);
>
> - unsigned footerBits = ((posSlot >> 1) - 1);
>
> + unsigned footerBits = (posSlot >> 1) - 1;
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> + const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
>
> // tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base,
> footerBits, i - base, p->ProbPrices);
>
> -
>
> - const CLzmaProb *probs = p->posEncoders + base;
>
> UInt32 price = 0;
>
> unsigned m = 1;
>
> - unsigned symbol = i - base;
>
> + unsigned sym = i;
>
> + unsigned offset = (unsigned)1 << footerBits;
>
> + base += i;
>
> +
>
> + if (footerBits)
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> price += GET_PRICEa(probs[m], bit);
>
> m = (m << 1) + bit;
>
> }
>
> while (--footerBits);
>
> - tempPrices[i] = price;
>
> +
>
> + {
>
> + unsigned prob = probs[m];
>
> + tempPrices[base ] = price + GET_PRICEa_0(prob);
>
> + tempPrices[base + offset] = price + GET_PRICEa_1(prob);
>
> + }
>
> }
>
>
>
> - for (lenToPosState = 0; lenToPosState < kNumLenToPosStates;
> lenToPosState++)
>
> + for (lps = 0; lps < kNumLenToPosStates; lps++)
>
> {
>
> - unsigned posSlot;
>
> - const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
>
> - UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
>
> - unsigned distTableSize = p->distTableSize;
>
> - const CLzmaProb *probs = encoder;
>
> - for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
>
> + unsigned slot;
>
> + unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
>
> + UInt32 *posSlotPrices = p->posSlotPrices[lps];
>
> + const CLzmaProb *probs = p->posSlotEncoder[lps];
>
> +
>
> + for (slot = 0; slot < distTableSize2; slot++)
>
> {
>
> - // posSlotPrices[posSlot] = RcTree_GetPrice(encoder,
> kNumPosSlotBits, posSlot, p->ProbPrices);
>
> - UInt32 price = 0;
>
> + // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits,
> slot, p->ProbPrices);
>
> + UInt32 price;
>
> unsigned bit;
>
> - unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
>
> - UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
>
> - posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
>
> - posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
>
> + unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
>
> + unsigned prob;
>
> + bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
>
> + posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
>
> + posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
>
> }
>
> - for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
>
> - posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
>
>
> {
>
> - UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
>
> + UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> + for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
>
> {
>
> - distancesPrices[0] = posSlotPrices[0];
>
> - distancesPrices[1] = posSlotPrices[1];
>
> - distancesPrices[2] = posSlotPrices[2];
>
> - distancesPrices[3] = posSlotPrices[3];
>
> + posSlotPrices[(size_t)slot * 2 ] += delta;
>
> + posSlotPrices[(size_t)slot * 2 + 1] += delta;
>
> + delta += ((UInt32)1 << kNumBitPriceShiftBits);
>
> }
>
> + }
>
> +
>
> + {
>
> + UInt32 *dp = p->distancesPrices[lps];
>
> +
>
> + dp[0] = posSlotPrices[0];
>
> + dp[1] = posSlotPrices[1];
>
> + dp[2] = posSlotPrices[2];
>
> + dp[3] = posSlotPrices[3];
>
> +
>
> for (i = 4; i < kNumFullDistances; i += 2)
>
> {
>
> UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
>
> - distancesPrices[i ] = slotPrice + tempPrices[i];
>
> - distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
>
> + dp[i ] = slotPrice + tempPrices[i];
>
> + dp[i + 1] = slotPrice + tempPrices[i + 1];
>
> }
>
> }
>
> }
>
> + // }
>
> }
>
>
>
>
>
> @@ -2037,7 +2202,7 @@ void LzmaEnc_Construct(CLzmaEnc *p)
> {
>
> RangeEnc_Construct(&p->rc);
>
> MatchFinder_Construct(&p->matchFinderBase);
>
> -
>
> +
>
> #ifndef _7ZIP_ST
>
> MatchFinderMt_Construct(&p->matchFinderMt);
>
> p->matchFinderMt.MatchFinder = &p->matchFinderBase;
>
> @@ -2081,7 +2246,7 @@ void LzmaEnc_Destruct(CLzmaEnc *p, ISzAllocPtr
> alloc, ISzAllocPtr allocBig)
> #ifndef _7ZIP_ST
>
> MatchFinderMt_Destruct(&p->matchFinderMt, allocBig);
>
> #endif
>
> -
>
> +
>
> MatchFinder_Free(&p->matchFinderBase, allocBig);
>
> LzmaEnc_FreeLits(p, alloc);
>
> RangeEnc_Free(&p->rc, alloc);
>
> @@ -2126,14 +2291,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> }
>
>
>
> if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) != 0)
>
> -
>
> +
>
> for (;;)
>
> {
>
> UInt32 dist;
>
> unsigned len, posState;
>
> UInt32 range, ttt, newBound;
>
> CLzmaProb *probs;
>
> -
>
> +
>
> if (p->fastMode)
>
> len = GetOptimumFast(p);
>
> else
>
> @@ -2153,9 +2318,9 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> posState = (unsigned)nowPos32 & p->pbMask;
>
> range = p->rc.range;
>
> probs = &p->isMatch[p->state][posState];
>
> -
>
> +
>
> RC_BIT_PRE(&p->rc, probs)
>
> -
>
> +
>
> dist = p->backRes;
>
>
>
> #ifdef SHOW_STAT2
>
> @@ -2185,7 +2350,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> RC_BIT_1(&p->rc, probs);
>
> probs = &p->isRep[p->state];
>
> RC_BIT_PRE(&p->rc, probs)
>
> -
>
> +
>
> if (dist < LZMA_NUM_REPS)
>
> {
>
> RC_BIT_1(&p->rc, probs);
>
> @@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> if (len != 1)
>
> {
>
> LenEnc_Encode(&p->repLenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->repLenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->repLenEnc, posState,
> &p->repLenProbs, p->ProbPrices);
>
> -
>
> + --p->repLenEncCounter;
>
> p->state = kRepNextStates[p->state];
>
> }
>
> }
>
> @@ -2260,34 +2422,32 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> p->state = kMatchNextStates[p->state];
>
>
>
> LenEnc_Encode(&p->lenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->lenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->lenEnc, posState,
> &p->lenProbs, p->ProbPrices);
>
> + // --p->lenEnc.counter;
>
>
>
> dist -= LZMA_NUM_REPS;
>
> p->reps[3] = p->reps[2];
>
> p->reps[2] = p->reps[1];
>
> p->reps[1] = p->reps[0];
>
> p->reps[0] = dist + 1;
>
> -
>
> +
>
> p->matchPriceCount++;
>
> GetPosSlot(dist, posSlot);
>
> // RcTree_Encode_PosSlot(&p->rc,
> p->posSlotEncoder[GetLenToPosState(len)], posSlot);
>
> {
>
> - UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
>
> + UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
>
> range = p->rc.range;
>
> probs = p->posSlotEncoder[GetLenToPosState(len)];
>
> do
>
> {
>
> - CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
>
> - UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
>
> - symbol <<= 1;
>
> + CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
>
> + UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
>
> + sym <<= 1;
>
> RC_BIT(&p->rc, prob, bit);
>
> }
>
> - while (symbol < (1 << kNumPosSlotBits * 2));
>
> + while (sym < (1 << kNumPosSlotBits * 2));
>
> p->rc.range = range;
>
> }
>
> -
>
> +
>
> if (dist >= kStartPosModelIndex)
>
> {
>
> unsigned footerBits = ((posSlot >> 1) - 1);
>
> @@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> if (dist < kNumFullDistances)
>
> {
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> - RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, dist - base);
>
> + RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, (unsigned)(dist /* - base */));
>
> }
>
> else
>
> {
>
> @@ -2331,32 +2491,49 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> bit = dist & 1; dist >>= 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit); m = (m << 1) + bit;
>
> bit = dist & 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit);
>
> p->rc.range = range;
>
> - p->alignPriceCount++;
>
> + // p->alignPriceCount++;
>
> }
>
> }
>
> }
>
> }
>
> }
>
>
>
> - nowPos32 += len;
>
> + nowPos32 += (UInt32)len;
>
> p->additionalOffset -= len;
>
> -
>
> +
>
> if (p->additionalOffset == 0)
>
> {
>
> UInt32 processed;
>
>
>
> if (!p->fastMode)
>
> {
>
> - if (p->matchPriceCount >= (1 << 7))
>
> + /*
>
> + if (p->alignPriceCount >= 16) // kAlignTableSize
>
> + FillAlignPrices(p);
>
> + if (p->matchPriceCount >= 128)
>
> FillDistancesPrices(p);
>
> - if (p->alignPriceCount >= kAlignTableSize)
>
> + if (p->lenEnc.counter <= 0)
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + */
>
> + if (p->matchPriceCount >= 64)
>
> + {
>
> FillAlignPrices(p);
>
> + // { int y; for (y = 0; y < 100; y++) {
>
> + FillDistancesPrices(p);
>
> + // }}
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + }
>
> + if (p->repLenEncCounter <= 0)
>
> + {
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> + LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb,
> &p->repLenProbs, p->ProbPrices);
>
> + }
>
> }
>
> -
>
> +
>
> if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
>
> break;
>
> processed = nowPos32 - startPos32;
>
> -
>
> +
>
> if (maxPackSize)
>
> {
>
> if (processed + kNumOpts + 300 >= maxUnpackSize
>
> @@ -2430,7 +2607,7 @@ static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32
> keepWindowSize, ISzAllocPtr alloc,
> p->matchFinderObj = &p->matchFinderBase;
>
> MatchFinder_CreateVTable(&p->matchFinderBase, &p->matchFinder);
>
> }
>
> -
>
> +
>
> return SZ_OK;
>
> }
>
>
>
> @@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
>
>
> p->optEnd = 0;
>
> p->optCur = 0;
>
> +
>
> + {
>
> + for (i = 0; i < kNumOpts; i++)
>
> + p->opt[i].price = kInfinityPrice;
>
> + }
>
> +
>
> p->additionalOffset = 0;
>
>
>
> p->pbMask = (1 << p->pb) - 1;
>
> p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
>
> }
>
>
>
> +
>
> void LzmaEnc_InitPrices(CLzmaEnc *p)
>
> {
>
> if (!p->fastMode)
>
> @@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
> p->lenEnc.tableSize =
>
> p->repLenEnc.tableSize =
>
> p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
>
> +
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> +
>
> LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs,
> p->ProbPrices);
>
> LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs,
> p->ProbPrices);
>
> }
>
> @@ -2583,7 +2770,7 @@ typedef struct
> ISeqOutStream vt;
>
> Byte *data;
>
> SizeT rem;
>
> - Bool overflow;
>
> + BoolInt overflow;
>
> } CLzmaEnc_SeqOutStreamBuf;
>
>
>
> static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void
> *data, size_t size)
>
> @@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle
> pp)
> }
>
>
>
>
>
> -SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
>
> +SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
>
> Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32
> *unpackSize)
>
> {
>
> CLzmaEnc *p = (CLzmaEnc *)pp;
>
> @@ -2644,7 +2831,7 @@ SRes
> LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
> return SZ_ERROR_OUTPUT_EOF;
>
>
>
> res = LzmaEnc_CodeOneBlock(p, desiredPackSize, *unpackSize);
>
> -
>
> +
>
> *unpackSize = (UInt32)(p->nowPos64 - nowPos64);
>
> *destLen -= outStream.rem;
>
> if (outStream.overflow)
>
> @@ -2679,7 +2866,7 @@ static SRes LzmaEnc_Encode2(CLzmaEnc *p,
> ICompressProgress *progress)
> }
>
> }
>
> }
>
> -
>
> +
>
> LzmaEnc_Finish(p);
>
>
>
> /*
>
> @@ -2751,7 +2938,7 @@ SRes LzmaEnc_MemEncode(CLzmaEncHandle pp,
> Byte *dest, SizeT *destLen, const Byte
> p->rc.outStream = &outStream.vt;
>
>
>
> res = LzmaEnc_MemPrepare(pp, src, srcLen, 0, alloc, allocBig);
>
> -
>
> +
>
> if (res == SZ_OK)
>
> {
>
> res = LzmaEnc_Encode2(p, progress);
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> index d7426d38a4..0f9560c7a6 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> @@ -1,13 +1,35 @@
> HISTORY of the LZMA SDK
>
> -----------------------
>
>
>
> +19.00 2019-02-21
>
> +-------------------------
>
> +- Encryption strength for 7z archives was increased:
>
> + the size of random initialization vector was increased from 64-bit to
> 128-bit,
>
> + and the pseudo-random number generator was improved.
>
> +- The bug in 7zIn.c code was fixed.
>
> +
>
> +
>
> +18.06 2018-12-30
>
> +-------------------------
>
> +- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
>
> + and there are minor changes in compression ratio.
>
> +- Some bugs were fixed.
>
> +- The bug in 7-Zip 18.02-18.05 was fixed:
>
> + There was memory leak in multithreading xz decoder -
> XzDecMt_Decode(),
>
> + if xz stream contains only one block.
>
> +- The changes for MSVS compiler makefiles:
>
> + - the makefiles now use "PLATFORM" macroname with values (x64, x86,
> arm64)
>
> + instead of "CPU" macroname with values (AMD64, ARM64).
>
> + - the makefiles by default now use static version of the run-time
library.
>
> +
>
> +
>
> 18.05 2018-04-30
>
> -------------------------
>
> -- The speed for LZMA/LZMA2 compressing was increased
>
> - by 8% for fastest/fast compression levels and
>
> +- The speed for LZMA/LZMA2 compressing was increased
>
> + by 8% for fastest/fast compression levels and
>
> by 3% for normal/maximum compression levels.
>
> - Previous versions of 7-Zip could work incorrectly in "Large memory
pages"
> mode in
>
> - Windows 10 because of some BUG with "Large Pages" in Windows 10.
>
> + Windows 10 because of some BUG with "Large Pages" in Windows 10.
>
> Now 7-Zip doesn't use "Large Pages" on Windows 10 up to revision 1709
> (16299).
>
> - The BUG was fixed in Lzma2Enc.c
>
> Lzma2Enc_Encode2() function worked incorretly,
>
> @@ -16,7 +38,7 @@ HISTORY of the LZMA SDK
>
>
> 18.03 beta 2018-03-04
>
> -------------------------
>
> -- Asm\x86\LzmaDecOpt.asm: new optimized LZMA decoder written in asm
>
> +- Asm\x86\LzmaDecOpt.asm: new optimized LZMA decoder written in asm
>
> for x64 with about 30% higher speed than main version of LZMA decoder
> written in C.
>
> - The speed for single-thread LZMA/LZMA2 decoder written in C was
> increased by 3%.
>
> - 7-Zip now can use multi-threading for 7z/LZMA2 decoding,
>
> @@ -35,7 +57,7 @@ HISTORY of the LZMA SDK
> 18.00 beta 2019-01-10
>
> -------------------------
>
> - The BUG in xz encoder was fixed:
>
> - There was memory leak of 16 KB for each file compressed with
>
> + There was memory leak of 16 KB for each file compressed with
>
> xz compression method, if additional filter was used.
>
>
>
>
>
> @@ -44,7 +66,7 @@ HISTORY of the LZMA SDK
> - Minor speed optimization for LZMA2 (xz and 7z) multi-threading
> compression.
>
> 7-Zip now uses additional memory buffers for multi-block LZMA2
> compression.
>
> CPU utilization was slightly improved.
>
> -- 7-zip now creates multi-block xz archives by default. Block size can be
>
> +- 7-zip now creates multi-block xz archives by default. Block size can be
>
> specified with -ms[Size]{m|g} switch.
>
> - xz decoder now can unpack random block from multi-block xz archives.
>
> - 7-Zip command line: @listfile now doesn't work after -- switch.
>
> @@ -55,7 +77,7 @@ HISTORY of the LZMA SDK
>
>
> 17.00 beta 2017-04-29
>
> -------------------------
>
> -- NewHandler.h / NewHandler.cpp:
>
> +- NewHandler.h / NewHandler.cpp:
>
> now it redefines operator new() only for old MSVC compilers
> (_MSC_VER < 1900).
>
> - C/7zTypes.h : the names of variables in interface structures were
changed
> (vt).
>
> - Some bugs were fixed. 7-Zip could crash in some cases.
>
> @@ -76,51 +98,51 @@ HISTORY of the LZMA SDK
> 16.02 2016-05-21
>
> -------------------------
>
> - The BUG in 16.00 - 16.01 was fixed:
>
> - Split Handler (SplitHandler.cpp) returned incorrect
>
> + Split Handler (SplitHandler.cpp) returned incorrect
>
> total size value (kpidSize) for split archives.
>
>
>
>
>
> 16.01 2016-05-19
>
> --------------------------
>
> +-------------------------
>
> - Some internal changes to reduce the number of compiler warnings.
>
>
>
>
>
> 16.00 2016-05-10
>
> --------------------------
>
> +-------------------------
>
> - Some bugs were fixed.
>
>
>
>
>
> 15.12 2015-11-19
>
> --------------------------
>
> +-------------------------
>
> - The BUG in C version of 7z decoder was fixed:
>
> 7zDec.c : SzDecodeLzma2()
>
> 7z decoder could mistakenly report about decoding error for some 7z
> archives
>
> that use LZMA2 compression method.
>
> - The probability to get that mistaken decoding error report was about
>
> - one error per 16384 solid blocks for solid blocks larger than 16 KB
> (compressed size).
>
> + The probability to get that mistaken decoding error report was about
>
> + one error per 16384 solid blocks for solid blocks larger than 16 KB
> (compressed size).
>
> - The BUG (in 9.26-15.11) in C version of 7z decoder was fixed:
>
> 7zArcIn.c : SzReadHeader2()
>
> - 7z decoder worked incorrectly for 7z archives that contain
>
> - empty solid blocks, that can be placed to 7z archive, if some file is
>
> + 7z decoder worked incorrectly for 7z archives that contain
>
> + empty solid blocks, that can be placed to 7z archive, if some file is
>
> unavailable for reading during archive creation.
>
>
>
>
>
> 15.09 beta 2015-10-16
>
> --------------------------
>
> +-------------------------
>
> - The BUG in LZMA / LZMA2 encoding code was fixed.
>
> The BUG in LzFind.c::MatchFinder_ReadBlock() function.
>
> If input data size is larger than (4 GiB - dictionary_size),
>
> the following code worked incorrectly:
>
> - - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding
> functions
>
> - for compressing from memory to memory.
>
> + - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding
> functions
>
> + for compressing from memory to memory.
>
> That BUG is not related to LZMA encoder version that works via
> streams.
>
> - - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly,
> if
>
> - default value of chunk size (CLzma2EncProps::blockSize) is changed
>
> + - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly,
> if
>
> + default value of chunk size (CLzma2EncProps::blockSize) is changed
>
> to value larger than (4 GiB - dictionary_size).
>
>
>
>
>
> 9.38 beta 2015-01-03
>
> --------------------------
>
> +-------------------------
>
> - The BUG in 9.31-9.37 was fixed:
>
> IArchiveGetRawProps interface was disabled for 7z archives.
>
> - The BUG in 9.26-9.36 was fixed:
>
> @@ -128,10 +150,10 @@ HISTORY of the LZMA SDK
>
>
>
>
> 9.36 beta 2014-12-26
>
> --------------------------
>
> +-------------------------
>
> - The BUG in command line version was fixed:
>
> 7-Zip created temporary archive in current folder during update archive
>
> - operation, if -w{Path} switch was not specified.
>
> + operation, if -w{Path} switch was not specified.
>
> The fixed 7-Zip creates temporary archive in folder that contains
updated
> archive.
>
> - The BUG in 9.33-9.35 was fixed:
>
> 7-Zip silently ignored file reading errors during 7z or gz archive
creation,
>
> @@ -140,14 +162,14 @@ HISTORY of the LZMA SDK
>
>
>
>
> 9.35 beta 2014-12-07
>
> --------------------------
>
> +-------------------------
>
> - 7zr.exe now support AES encryption.
>
> - SFX modules were added to LZMA SDK
>
> - Some bugs were fixed.
>
>
>
>
>
> 9.21 beta 2011-04-11
>
> --------------------------
>
> +-------------------------
>
> - New class FString for file names at file systems.
>
> - Speed optimization in CRC code for big-endian CPUs.
>
> - The BUG in Lzma2Dec.c was fixed:
>
> @@ -155,7 +177,7 @@ HISTORY of the LZMA SDK
>
>
>
>
> 9.18 beta 2010-11-02
>
> --------------------------
>
> +-------------------------
>
> - New small SFX module for installers (SfxSetup).
>
>
>
>
>
> @@ -168,7 +190,7 @@ HISTORY of the LZMA SDK
> 9.11 beta 2010-03-15
>
> -------------------------
>
> - PPMd compression method support
>
> -
>
> +
>
>
>
> 9.09 2009-12-12
>
> -------------------------
>
> @@ -202,7 +224,7 @@ HISTORY of the LZMA SDK
> 4.61 beta 2008-11-23
>
> -------------------------
>
> - The bug in ANSI-C LZMA Decoder was fixed:
>
> - If encoded stream was corrupted, decoder could access memory
>
> + If encoded stream was corrupted, decoder could access memory
>
> outside of allocated range.
>
> - Some changes in ANSI-C 7z Decoder interfaces.
>
> - LZMA SDK is placed in the public domain.
>
> @@ -216,7 +238,7 @@ HISTORY of the LZMA SDK
> 4.59 beta 2008-08-13
>
> -------------------------
>
> - The bug was fixed:
>
> - LZMA Encoder in fast compression mode could access memory outside
> of
>
> + LZMA Encoder in fast compression mode could access memory outside
> of
>
> allocated range in some rare cases.
>
>
>
>
>
> @@ -229,7 +251,7 @@ HISTORY of the LZMA SDK
>
>
> 4.57 2007-12-12
>
> -------------------------
>
> -- Speed optimizations in ?++ LZMA Decoder.
>
> +- Speed optimizations in ?++ LZMA Decoder.
>
> - Small changes for more compatibility with some C/C++ compilers.
>
>
>
>
>
> @@ -239,36 +261,36 @@ HISTORY of the LZMA SDK
> - now it supports BCJ and BCJ2 filters
>
> - now it supports files larger than 4 GB.
>
> - now it supports "Last Write Time" field for files.
>
> -- C++ code for .7z archives compressing/decompressing from 7-zip
>
> +- C++ code for .7z archives compressing/decompressing from 7-zip
>
> was included to LZMA SDK.
>
> -
>
> +
>
>
>
> 4.43 2006-06-04
>
> -------------------------
>
> - Small changes for more compatibility with some C/C++ compilers.
>
> -
>
> +
>
>
>
> 4.42 2006-05-15
>
> -------------------------
>
> - Small changes in .h files in ANSI-C version.
>
> -
>
> +
>
>
>
> 4.39 beta 2006-04-14
>
> -------------------------
>
> - The bug in versions 4.33b:4.38b was fixed:
>
> - C++ version of LZMA encoder could not correctly compress
>
> + C++ version of LZMA encoder could not correctly compress
>
> files larger than 2 GB with HC4 match finder (-mfhc4).
>
> -
>
> +
>
>
>
> 4.37 beta 2005-04-06
>
> -------------------------
>
> -- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was
> defined.
>
> +- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was
> defined.
>
>
>
>
>
> 4.35 beta 2005-03-02
>
> -------------------------
>
> - The bug was fixed in C++ version of LZMA Decoder:
>
> - If encoded stream was corrupted, decoder could access memory
>
> + If encoded stream was corrupted, decoder could access memory
>
> outside of allocated range.
>
>
>
>
>
> @@ -339,7 +361,7 @@ HISTORY of the LZMA SDK
>
>
> 4.17 2005-04-18
>
> -------------------------
>
> -- New example for RAM->RAM compressing/decompressing:
>
> +- New example for RAM->RAM compressing/decompressing:
>
> LZMA + BCJ (filter for x86 code):
>
> - LzmaRam.h
>
> - LzmaRam.cpp
>
> @@ -350,11 +372,11 @@ HISTORY of the LZMA SDK
>
>
> 4.16 2005-03-29
>
> -------------------------
>
> -- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):
>
> +- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):
>
> If _LZMA_OUT_READ was defined, and if encoded stream was corrupted,
>
> decoder could access memory outside of allocated range.
>
> - Speed optimization of ANSI-C LZMA Decoder (now it's about 20% faster).
>
> - Old version of LZMA Decoder now is in file LzmaDecodeSize.c.
>
> + Old version of LZMA Decoder now is in file LzmaDecodeSize.c.
>
> LzmaDecodeSize.c can provide slightly smaller code than LzmaDecode.c
>
> - Small speed optimization in LZMA C++ code
>
> - filter for SPARC's code was added
>
> @@ -369,7 +391,7 @@ HISTORY of the LZMA SDK
>
>
> 4.05 2004-08-25
>
> -------------------------
>
> -- Source code of filters for x86, IA-64, ARM, ARM-Thumb
>
> +- Source code of filters for x86, IA-64, ARM, ARM-Thumb
>
> and PowerPC code was included to SDK
>
> - Some internal minor changes
>
>
>
> @@ -381,8 +403,8 @@ HISTORY of the LZMA SDK
>
>
> 4.03 2004-06-18
>
> -------------------------
>
> -- "Benchmark" command was added. It measures compressing
>
> - and decompressing speed and shows rating values.
>
> +- "Benchmark" command was added. It measures compressing
>
> + and decompressing speed and shows rating values.
>
> Also it checks hardware errors.
>
>
>
>
>
> @@ -411,7 +433,7 @@ HISTORY of the LZMA SDK
>
>
> HISTORY of the LZMA
>
> -------------------
>
> - 2001-2008: Improvements to LZMA compressing/decompressing code,
>
> + 2001-2008: Improvements to LZMA compressing/decompressing code,
>
> keeping compatibility with original LZMA format
>
> 1996-2001: Development of LZMA compression format
>
>
>
> @@ -419,6 +441,6 @@ HISTORY of the LZMA
>
>
> 2001-08-30: LZMA compression was added to 7-Zip
>
> 1999-01-02: First version of 7-Zip was released
>
> -
>
> +
>
>
>
> End of document
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> index a3deb20943..e17cd4fe1a 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> @@ -1,4 +1,4 @@
> -LZMA SDK 18.05
>
> +LZMA SDK 19.00
>
> --------------
>
>
>
> LZMA SDK provides the documentation, samples, header files,
>
> --
> 2.29.2.windows.3
>
>
>
> -=-=-=-=-=-=
> Groups.io Links: You receive all messages sent to this group.
> View/Reply Online (#69675): https://edk2.groups.io/g/devel/message/69675
> Mute This Topic: https://groups.io/mt/79445451/4905953
> Group Owner: devel+owner@edk2.groups.io
> Unsubscribe: https://edk2.groups.io/g/devel/unsub
> [gaoliming@byosoft.com.cn]
> -=-=-=-=-=-=
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
@ 2020-12-29 7:37 Liu, WeiX C
0 siblings, 0 replies; 7+ messages in thread
From: Liu, WeiX C @ 2020-12-29 7:37 UTC (permalink / raw)
To: devel; +Cc: Wei Liu, Liming Gao
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
New formal release in https://www.7-zip.org/sdk.html is 19.00.
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Wei Liu <weix.c.liu@intel.com>
---
BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4 ++--
BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8 ++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5 +++--
BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8 ++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9 +++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 232 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14 +++++++-------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32 ++++++++++++++++----------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 907 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 114 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
12 files changed, 877 insertions(+), 575 deletions(-)
diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
index 7cf967a774..b2338e2598 100644
--- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
+++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
@@ -1,3 +1,3 @@
-LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
-was placed in the public domain on 2018-04-30. It was
+LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
+was placed in the public domain on 2019-02-21. It was
released on the http://www.7-zip.org/sdk.html website.
diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
index bebdb9aa84..bf4706ee23 100644
--- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
+++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
@@ -1,9 +1,9 @@
/** @file
LZMA Compress/Decompress tool (LzmaCompress)
- Based on LZMA SDK 18.05:
+ Based on LZMA SDK 19.00:
LzmaUtil.c -- Test application for LZMA compression
- 2018-04-30 : Igor Pavlov : Public domain
+ 2019-02-21 : Igor Pavlov : Public domain
Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
SPDX-License-Identifier: BSD-2-Clause-Patent
@@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate memory";
const char *kDataErrorMessage = "Data error";
const char *kInvalidParamValMessage = "Invalid parameter value";
-static Bool mQuietMode = False;
+static BoolInt mQuietMode = False;
static CONVERTER_TYPE mConType = NoConverter;
UINT64 mDictionarySize = 28;
@@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
CFileOutStream outStream;
int res;
int encodeMode = 0;
- Bool modeWasSet = False;
+ BoolInt modeWasSet = False;
const char *inputFile = NULL;
const char *outputFile = "file.tmp";
int param;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
index 4977cdaa66..593f5aa259 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
@@ -1,5 +1,5 @@
/* 7zTypes.h -- Basic types
-2017-07-17 : Igor Pavlov : Public domain */
+2018-08-04 : Igor Pavlov : Public domain */
#ifndef __7Z_TYPES_H
#define __7Z_TYPES_H
@@ -103,7 +103,8 @@ typedef UInt32 SizeT;
typedef size_t SizeT;
#endif
-typedef int Bool;
+typedef int BoolInt;
+/* typedef BoolInt Bool; */
#define True 1
#define False 0
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
index ed3aa94270..0074c64be9 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
@@ -1,7 +1,7 @@
-#define MY_VER_MAJOR 18
-#define MY_VER_MINOR 05
+#define MY_VER_MAJOR 19
+#define MY_VER_MINOR 00
#define MY_VER_BUILD 0
-#define MY_VERSION_NUMBERS "18.05"
+#define MY_VERSION_NUMBERS "19.00"
#define MY_VERSION MY_VERSION_NUMBERS
#ifdef MY_CPU_NAME
@@ -10,7 +10,7 @@
#define MY_VERSION_CPU MY_VERSION
#endif
-#define MY_DATE "2018-04-30"
+#define MY_DATE "2019-02-21"
#undef MY_COPYRIGHT
#undef MY_VERSION_COPYRIGHT_DATE
#define MY_AUTHOR_NAME "Igor Pavlov"
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
index 7fb27282c7..5f74c1c0cb 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
@@ -1,5 +1,5 @@
/* CpuArch.h -- CPU specific code
-2017-09-04 : Igor Pavlov : Public domain */
+2018-02-18 : Igor Pavlov : Public domain */
#ifndef __CPU_ARCH_H
#define __CPU_ARCH_H
@@ -318,15 +318,16 @@ enum
void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d);
-Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
+BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
int x86cpuid_GetFirm(const Cx86cpuid *p);
#define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) & 0xF))
#define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) & 0xF))
#define x86cpuid_GetStepping(ver) (ver & 0xF)
-Bool CPU_Is_InOrder();
-Bool CPU_Is_Aes_Supported();
+BoolInt CPU_Is_InOrder();
+BoolInt CPU_Is_Aes_Supported();
+BoolInt CPU_IsSupported_PageGB();
#endif
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
index 6ea82a9b53..f89c9ddd7d 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
@@ -1,5 +1,5 @@
/* LzFind.c -- Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-07-08 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -73,7 +73,7 @@ static void MatchFinder_ReadBlock(CMatchFinder *p)
p->streamEndWasReached = 1;
return;
}
-
+
for (;;)
{
Byte *dest = p->buffer + (p->streamPos - p->pos);
@@ -138,7 +138,7 @@ static void MatchFinder_SetDefaultSettings(CMatchFinder *p)
void MatchFinder_Construct(CMatchFinder *p)
{
- UInt32 i;
+ unsigned i;
p->bufferBase = NULL;
p->directInput = 0;
p->hash = NULL;
@@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
for (i = 0; i < 256; i++)
{
- UInt32 r = i;
+ UInt32 r = (UInt32)i;
unsigned j;
for (j = 0; j < 8; j++)
r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
@@ -180,24 +180,24 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
ISzAllocPtr alloc)
{
UInt32 sizeReserv;
-
+
if (historySize > kMaxHistorySize)
{
MatchFinder_Free(p, alloc);
return 0;
}
-
+
sizeReserv = historySize >> 1;
if (historySize >= ((UInt32)3 << 30)) sizeReserv = historySize >> 3;
else if (historySize >= ((UInt32)2 << 30)) sizeReserv = historySize >> 2;
-
+
sizeReserv += (keepAddBufferBefore + matchMaxLen + keepAddBufferAfter) / 2 + (1 << 19);
p->keepSizeBefore = historySize + keepAddBufferBefore + 1;
p->keepSizeAfter = matchMaxLen + keepAddBufferAfter;
-
+
/* we need one additional byte, since we use MoveBlock after pos++ and before dictionary using */
-
+
if (LzInWindow_Create(p, sizeReserv, alloc))
{
UInt32 newCyclicBufferSize = historySize + 1;
@@ -243,7 +243,7 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
p->historySize = historySize;
p->hashSizeSum = hs;
p->cyclicBufferSize = newCyclicBufferSize;
-
+
numSons = newCyclicBufferSize;
if (p->btMode)
numSons <<= 1;
@@ -251,11 +251,11 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
if (p->hash && p->numRefs == newSize)
return 1;
-
+
MatchFinder_FreeThisClassMemory(p, alloc);
p->numRefs = newSize;
p->hash = AllocRefs(newSize, alloc);
-
+
if (p->hash)
{
p->son = p->hash + p->hashSizeSum;
@@ -272,11 +272,11 @@ static void MatchFinder_SetLimits(CMatchFinder *p)
{
UInt32 limit = kMaxValForNormalize - p->pos;
UInt32 limit2 = p->cyclicBufferSize - p->cyclicBufferPos;
-
+
if (limit2 < limit)
limit = limit2;
limit2 = p->streamPos - p->pos;
-
+
if (limit2 <= p->keepSizeAfter)
{
if (limit2 > 0)
@@ -284,10 +284,10 @@ static void MatchFinder_SetLimits(CMatchFinder *p)
}
else
limit2 -= p->keepSizeAfter;
-
+
if (limit2 < limit)
limit = limit2;
-
+
{
UInt32 lenLimit = p->streamPos - p->pos;
if (lenLimit > p->matchMaxLen)
@@ -326,10 +326,10 @@ void MatchFinder_Init_3(CMatchFinder *p, int readData)
p->streamPos = p->cyclicBufferSize;
p->result = SZ_OK;
p->streamEndWasReached = 0;
-
+
if (readData)
MatchFinder_ReadBlock(p);
-
+
MatchFinder_SetLimits(p);
}
@@ -341,7 +341,7 @@ void MatchFinder_Init(CMatchFinder *p)
MatchFinder_Init_3(p, True);
}
-
+
static UInt32 MatchFinder_GetSubValue(CMatchFinder *p)
{
return (p->pos - p->historySize - 1) & kNormalizeMask;
@@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
MatchFinder_ReduceOffsets(p, subValue);
}
+
+MY_NO_INLINE
static void MatchFinder_CheckLimits(CMatchFinder *p)
{
if (p->pos == kMaxValForNormalize)
@@ -379,10 +381,16 @@ static void MatchFinder_CheckLimits(CMatchFinder *p)
MatchFinder_SetLimits(p);
}
-static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
+
+/*
+ (lenLimit > maxLen)
+*/
+MY_FORCE_INLINE
+static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
- UInt32 *distances, UInt32 maxLen)
+ UInt32 *distances, unsigned maxLen)
{
+ /*
son[_cyclicBufferPos] = curMatch;
for (;;)
{
@@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = len;
*distances++ = delta - 1;
if (len == lenLimit)
return distances;
@@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
}
}
}
+ */
+
+ const Byte *lim = cur + lenLimit;
+ son[_cyclicBufferPos] = curMatch;
+ do
+ {
+ UInt32 delta = pos - curMatch;
+ if (delta >= _cyclicBufferSize)
+ break;
+ {
+ ptrdiff_t diff;
+ curMatch = son[_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)];
+ diff = (ptrdiff_t)0 - delta;
+ if (cur[maxLen] == cur[maxLen + diff])
+ {
+ const Byte *c = cur;
+ while (*c == c[diff])
+ {
+ if (++c == lim)
+ {
+ distances[0] = (UInt32)(lim - cur);
+ distances[1] = delta - 1;
+ return distances + 2;
+ }
+ }
+ {
+ unsigned len = (unsigned)(c - cur);
+ if (maxLen < len)
+ {
+ maxLen = len;
+ distances[0] = (UInt32)len;
+ distances[1] = delta - 1;
+ distances += 2;
+ }
+ }
+ }
+ }
+ }
+ while (--cutValue);
+
+ return distances;
}
+
+MY_FORCE_INLINE
UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
UInt32 *distances, UInt32 maxLen)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
return distances;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = pair[0];
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = (UInt32)len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
+ *ptr1 = pair0;
*ptr0 = pair[1];
return distances;
}
@@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
return;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
if (pb[len] == cur[len])
{
while (++len != lenLimit)
@@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
p->buffer++; \
if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
-#define MOVE_POS_RET MOVE_POS return offset;
+#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define GET_MATCHES_HEADER2(minLen, ret_op) \
- UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
- lenLimit = p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
+ unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
+ lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
cur = p->buffer;
#define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen, return 0)
@@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos, p->cyclicBufferSize, p->cutValue
#define GET_MATCHES_FOOTER(offset, maxLen) \
- offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
- distances + offset, maxLen) - distances); MOVE_POS_RET;
+ offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch, MF_PARAMS(p), \
+ distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
#define SKIP_FOOTER \
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
#define UPDATE_maxLen { \
ptrdiff_t diff = (ptrdiff_t)0 - d2; \
const Byte *c = cur + maxLen; \
const Byte *lim = cur + lenLimit; \
for (; c != lim; c++) if (*(c + diff) != *c) break; \
- maxLen = (UInt32)(c - cur); }
+ maxLen = (unsigned)(c - cur); }
static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(2)
HASH2_CALC;
curMatch = p->hash[hv];
@@ -561,7 +615,7 @@ static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
@@ -572,7 +626,8 @@ UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, d2, maxLen, offset, pos;
+ UInt32 h2, d2, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(3)
@@ -584,7 +639,7 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
d2 = pos - hash[h2];
curMatch = (hash + kFix3HashSize)[hv];
-
+
hash[h2] = pos;
(hash + kFix3HashSize)[hv] = pos;
@@ -594,22 +649,23 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
UPDATE_maxLen
- distances[0] = maxLen;
+ distances[0] = (UInt32)maxLen;
distances[1] = d2 - 1;
offset = 2;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
-
+
GET_MATCHES_FOOTER(offset, maxLen)
}
static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -618,25 +674,26 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[hv] = pos;
maxLen = 0;
offset = 0;
-
+
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
-
+
if (d2 != d3 && d3 < p->cyclicBufferSize && *(cur - d3) == *cur)
{
maxLen = 3;
@@ -644,21 +701,21 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d3;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
-
+
if (maxLen < 3)
maxLen = 3;
-
+
GET_MATCHES_FOOTER(offset, maxLen)
}
@@ -674,13 +731,13 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
d4 = pos - (hash + kFix4HashSize)[h4];
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[h4] = pos;
(hash + kFix5HashSize)[hv] = pos;
@@ -710,7 +767,7 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset = 2;
d2 = d3;
}
-
+
if (d2 != d4 && d4 < p->cyclicBufferSize
&& *(cur - d4) == *cur
&& *(cur - d4 + 3) == *(cur + 3))
@@ -720,7 +777,7 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d4;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
@@ -734,14 +791,15 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (maxLen < 4)
maxLen = 4;
-
+
GET_MATCHES_FOOTER(offset, maxLen)
}
*/
static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -749,13 +807,12 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
-
- d2 = pos - hash[ h2];
+
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
-
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[hv] = pos;
@@ -764,11 +821,12 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
-
+
if (d2 != d3 && d3 < p->cyclicBufferSize && *(cur - d3) == *cur)
{
maxLen = 3;
@@ -776,22 +834,22 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d3;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
p->son[p->cyclicBufferPos] = curMatch;
MOVE_POS_RET;
}
}
-
+
if (maxLen < 3)
maxLen = 3;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances + offset, maxLen) - (distances));
MOVE_POS_RET
}
@@ -807,14 +865,14 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
-
- d2 = pos - hash[ h2];
+
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
d4 = pos - (hash + kFix4HashSize)[h4];
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[h4] = pos;
(hash + kFix5HashSize)[hv] = pos;
@@ -844,7 +902,7 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset = 2;
d2 = d3;
}
-
+
if (d2 != d4 && d4 < p->cyclicBufferSize
&& *(cur - d4) == *cur
&& *(cur - d4 + 3) == *(cur + 3))
@@ -854,7 +912,7 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
offset += 2;
d2 = d4;
}
-
+
if (offset != 0)
{
UPDATE_maxLen
@@ -865,7 +923,7 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
MOVE_POS_RET;
}
}
-
+
if (maxLen < 4)
maxLen = 4;
@@ -877,12 +935,12 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
p->hash[hv] = p->pos;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances, 2) - (distances));
MOVE_POS_RET
}
@@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH4_CALC;
hash = p->hash;
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[hv] = p->pos;
SKIP_FOOTER
@@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH5_CALC;
hash = p->hash;
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[h4] =
(hash + kFix5HashSize)[hv] = p->pos;
@@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH4_CALC;
hash = p->hash;
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[hv] = p->pos;
p->son[p->cyclicBufferPos] = curMatch;
@@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH5_CALC;
hash = p->hash;
curMatch = hash + kFix5HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[h4] =
(hash + kFix5HashSize)[hv] = p->pos;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
index 2563824fcd..df32146f92 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
@@ -1,5 +1,5 @@
/* LzFindMt.c -- multithreaded Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-12-29 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -232,38 +232,57 @@ static void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
#define kEmptyHashValue 0
-/* #define MFMT_GM_INLINE */
+#define MFMT_GM_INLINE
#ifdef MFMT_GM_INLINE
-#define NO_INLINE MY_FAST_CALL
+/*
+ we use size_t for _cyclicBufferPos instead of UInt32
+ to eliminate "movsx" BUG in old MSVC x64 compiler.
+*/
-static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
- UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
- UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit, UInt32 size, UInt32 *posRes)
+MY_NO_INLINE
+static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
+ size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
+ UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32 *limit, UInt32 size, UInt32 *posRes)
{
do
{
- UInt32 *distances = _distances + 1;
- UInt32 curMatch = pos - *hash++;
+ UInt32 *_distances = ++distances;
+ UInt32 delta = *hash++;
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
UInt32 cutValue = _cutValue;
- UInt32 maxLen = _maxLen;
- for (;;)
+ unsigned maxLen = (unsigned)_maxLen;
+
+ /*
+ if (size > 1)
{
- UInt32 delta = pos - curMatch;
- if (cutValue-- == 0 || delta >= _cyclicBufferSize)
+ UInt32 delta = *hash;
+ if (delta < _cyclicBufferSize)
{
- *ptr0 = *ptr1 = kEmptyHashValue;
- break;
+ UInt32 cyc1 = _cyclicBufferPos + 1;
+ CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ? _cyclicBufferSize : 0)) << 1);
+ Byte b = *(cur + 1 - delta);
+ _distances[0] = pair[0];
+ _distances[1] = b;
}
+ }
+ */
+ if (cutValue == 0 || delta >= _cyclicBufferSize)
+ {
+ *ptr0 = *ptr1 = kEmptyHashValue;
+ }
+ else
+ for(;;)
+ {
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = *pair;
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
- *ptr0 = pair[1];
+ UInt32 pair1 = pair[1];
+ *ptr1 = pair0;
+ *ptr0 = pair1;
break;
}
}
}
- if (pb[len] < cur[len])
- {
- *ptr1 = curMatch;
- ptr1 = pair + 1;
- curMatch = *ptr1;
- len1 = len;
- }
- else
{
- *ptr0 = curMatch;
- ptr0 = pair;
- curMatch = *ptr0;
- len0 = len;
+ UInt32 curMatch = pos - delta;
+ // delta = pos - *pair;
+ // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >> 31];
+ if (pb[len] < cur[len])
+ {
+ delta = pos - pair[1];
+ *ptr1 = curMatch;
+ ptr1 = pair + 1;
+ len1 = len;
+ }
+ else
+ {
+ delta = pos - *pair;
+ *ptr0 = curMatch;
+ ptr0 = pair;
+ len0 = len;
+ }
}
}
+ if (--cutValue == 0 || delta >= _cyclicBufferSize)
+ {
+ *ptr0 = *ptr1 = kEmptyHashValue;
+ break;
+ }
}
pos++;
_cyclicBufferPos++;
cur++;
{
UInt32 num = (UInt32)(distances - _distances);
- *_distances = num - 1;
- _distances += num;
- limit -= num;
+ _distances[-1] = num;
}
}
- while (limit > 0 && --size != 0);
+ while (distances < limit && --size != 0);
*posRes = pos;
- return limit;
+ return distances;
}
#endif
+
+
static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
{
UInt32 numProcessed = 0;
UInt32 curPos = 2;
- UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
+ UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
distances[1] = p->hashNumAvail;
@@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
#else
{
UInt32 posRes;
- curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
- distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
+ curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
+ distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos,
+ distances + limit,
+ size, &posRes) - distances);
p->hashBufPos += posRes - pos;
cyclicBufferPos += posRes - pos;
p->buffer += posRes - pos;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
index 3d86c788f3..fdd17008c2 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
@@ -1,5 +1,5 @@
/* LzFindMt.h -- multithreaded Match finder for LZ algorithms
-2017-04-03 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#ifndef __LZ_FIND_MT_H
#define __LZ_FIND_MT_H
@@ -19,10 +19,10 @@ EXTERN_C_BEGIN
typedef struct _CMtSync
{
- Bool wasCreated;
- Bool needStart;
- Bool exit;
- Bool stopWriting;
+ BoolInt wasCreated;
+ BoolInt needStart;
+ BoolInt exit;
+ BoolInt stopWriting;
CThread thread;
CAutoResetEvent canStart;
@@ -30,8 +30,8 @@ typedef struct _CMtSync
CAutoResetEvent wasStopped;
CSemaphore freeSemaphore;
CSemaphore filledSemaphore;
- Bool csWasInitialized;
- Bool csWasEntered;
+ BoolInt csWasInitialized;
+ BoolInt csWasEntered;
CCriticalSection cs;
UInt32 numProcessedBlocks;
} CMtSync;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
index 962b94bb63..4d1576419f 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
@@ -1,13 +1,13 @@
/* LzmaDec.c -- LZMA Decoder
-2018-02-28 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
+#include <string.h>
+
/* #include "CpuArch.h" */
#include "LzmaDec.h"
-#include <string.h>
-
#define kNumTopBits 24
#define kTopValue ((UInt32)1 << kNumTopBits)
@@ -19,7 +19,7 @@
#define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
#define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt - (ttt >> kNumMoveBits));
#define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
@@ -66,7 +66,7 @@
#define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit) return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0_CHECK range = bound;
#define UPDATE_1_CHECK range -= bound; code -= bound;
#define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
@@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
curLen = ((rem < len) ? (unsigned)rem : len);
pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
- processedPos += curLen;
+ processedPos += (UInt32)curLen;
len -= curLen;
if (curLen <= dicBufSize - pos)
@@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
Byte *dest = dic + dicPos;
ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
const Byte *lim = dest + curLen;
- dicPos += curLen;
+ dicPos += (SizeT)curLen;
do
*(dest) = (Byte)*(dest + src);
while (++dest != lim);
@@ -572,14 +572,14 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
p->buf = buf;
p->range = range;
p->code = code;
- p->remainLen = len;
+ p->remainLen = (UInt32)len;
p->dicPos = dicPos;
p->processedPos = processedPos;
p->reps[0] = rep0;
p->reps[1] = rep1;
p->reps[2] = rep2;
p->reps[3] = rep3;
- p->state = state;
+ p->state = (UInt32)state;
return SZ_OK;
}
@@ -601,8 +601,8 @@ static void MY_FAST_CALL LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
p->checkDicSize = p->prop.dicSize;
- p->processedPos += len;
- p->remainLen -= len;
+ p->processedPos += (UInt32)len;
+ p->remainLen -= (UInt32)len;
while (len != 0)
{
len--;
@@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, const Byte *buf, SizeT inS
}
-void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
+void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt initState)
{
p->remainLen = kMatchSpecLenStart + 1;
p->tempBufSize = 0;
@@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
p->tempBufSize = rem;
if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
{
- int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
+ int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, (SizeT)rem);
if (dummyRes == DUMMY_ERROR)
{
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
*status = LZMA_STATUS_NEEDS_MORE_INPUT;
return SZ_OK;
}
@@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
return SZ_ERROR_FAIL; /* some internal error */
lookAhead -= rem;
}
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
src += lookAhead;
- inSize -= lookAhead;
+ inSize -= (SizeT)lookAhead;
p->tempBufSize = 0;
}
}
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
index e281716fee..4e9b499f8d 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
@@ -1,5 +1,5 @@
/* LzmaEnc.c -- LZMA Encoder
-2018-04-29 : Igor Pavlov : Public domain */
+2019-01-10: Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
#define kNumBitPriceShiftBits 4
#define kBitPrice (1 << kNumBitPriceShiftBits)
+#define REP_LEN_COUNT 64
+
void LzmaEncProps_Init(CLzmaEncProps *p)
{
p->level = 5;
@@ -52,7 +54,7 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p)
int level = p->level;
if (level < 0) level = 5;
p->level = level;
-
+
if (p->dictSize == 0) p->dictSize = (level <= 5 ? (1 << (level * 2 + 14)) : (level <= 7 ? (1 << 25) : (1 << 26)));
if (p->dictSize > p->reduceSize)
{
@@ -74,7 +76,7 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p)
if (p->btMode < 0) p->btMode = (p->algo == 0 ? 0 : 1);
if (p->numHashBytes < 0) p->numHashBytes = 4;
if (p->mc == 0) p->mc = (16 + (p->fb >> 1)) >> (p->btMode ? 0 : 1);
-
+
if (p->numThreads < 0)
p->numThreads =
#ifndef _7ZIP_ST
@@ -124,7 +126,7 @@ static void LzmaEnc_FastPosInit(Byte *g_FastPos)
g_FastPos[0] = 0;
g_FastPos[1] = 1;
g_FastPos += 2;
-
+
for (slot = 2; slot < kNumLogBits * 2; slot++)
{
size_t k = ((size_t)1 << ((slot >> 1) - 1));
@@ -183,8 +185,11 @@ typedef struct
} COptimal;
-#define kNumOpts (1 << 12)
-#define kPackReserve (1 + kNumOpts * 2)
+// 18.06
+#define kNumOpts (1 << 11)
+#define kPackReserve (kNumOpts * 8)
+// #define kNumOpts (1 << 12)
+// #define kPackReserve (1 + kNumOpts * 2)
#define kNumLenToPosStates 4
#define kNumPosSlotBits 6
@@ -236,10 +241,18 @@ typedef struct
typedef struct
{
unsigned tableSize;
- unsigned counters[LZMA_NUM_PB_STATES_MAX];
UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
+ // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols * 2];
+ // UInt32 prices2[kLenNumSymbolsTotal];
} CLenPriceEnc;
+#define GET_PRICE_LEN(p, posState, len) \
+ ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
+
+/*
+#define GET_PRICE_LEN(p, posState, len) \
+ ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) - 2) & (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >> 9)))
+*/
typedef struct
{
@@ -273,7 +286,7 @@ typedef struct
CLzmaProb posSlotEncoder[kNumLenToPosStates][1 << kNumPosSlotBits];
CLzmaProb posEncoders[kNumFullDistances];
-
+
CLenEnc lenProbs;
CLenEnc repLenProbs;
@@ -308,16 +321,18 @@ typedef struct
unsigned lc, lp, pb;
unsigned lclp;
- Bool fastMode;
- Bool writeEndMark;
- Bool finished;
- Bool multiThread;
- Bool needInit;
+ BoolInt fastMode;
+ BoolInt writeEndMark;
+ BoolInt finished;
+ BoolInt multiThread;
+ BoolInt needInit;
+ // BoolInt _maxMode;
UInt64 nowPos64;
-
+
unsigned matchPriceCount;
- unsigned alignPriceCount;
+ // unsigned alignPriceCount;
+ int repLenEncCounter;
unsigned distTableSize;
@@ -325,7 +340,7 @@ typedef struct
SRes result;
#ifndef _7ZIP_ST
- Bool mtMode;
+ BoolInt mtMode;
// begin of CMatchFinderMt is used in LZ thread
CMatchFinderMt matchFinderMt;
// end of CMatchFinderMt is used in BT and HASH threads
@@ -336,7 +351,7 @@ typedef struct
#ifndef _7ZIP_ST
Byte pad[128];
#endif
-
+
// LZ thread
CProbPrice ProbPrices[kBitModelTotal >> kNumMoveReducingBits];
@@ -355,7 +370,7 @@ typedef struct
CLzmaProb isRep0Long[kNumStates][LZMA_NUM_PB_STATES_MAX];
CLzmaProb posSlotEncoder[kNumLenToPosStates][1 << kNumPosSlotBits];
CLzmaProb posEncoders[kNumFullDistances];
-
+
CLenEnc lenProbs;
CLenEnc repLenProbs;
@@ -383,9 +398,9 @@ void LzmaEnc_SaveState(CLzmaEncHandle pp)
{
CLzmaEnc *p = (CLzmaEnc *)pp;
CSaveState *dest = &p->saveState;
-
+
dest->state = p->state;
-
+
dest->lenProbs = p->lenProbs;
dest->repLenProbs = p->repLenProbs;
@@ -414,9 +429,9 @@ void LzmaEnc_RestoreState(CLzmaEncHandle pp)
dest->lenProbs = p->lenProbs;
dest->repLenProbs = p->repLenProbs;
-
+
COPY_ARR(dest, p, reps);
-
+
COPY_ARR(dest, p, posAlignEncoder);
COPY_ARR(dest, p, isRep);
COPY_ARR(dest, p, isRepG0);
@@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEncProps *props2)
p->lp = props.lp;
p->pb = props.pb;
p->fastMode = (props.algo == 0);
+ // p->_maxMode = True;
p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
{
unsigned numHashBytes = 4;
@@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
#ifdef _LZMA_ENC_USE_BRANCH
-#define RC_BIT(p, prob, symbol) { \
+#define RC_BIT(p, prob, bit) { \
RC_BIT_PRE(p, prob) \
- if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >> kNumMoveBits; } \
+ if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >> kNumMoveBits; } \
else { (p)->low += newBound; range -= newBound; ttt -= ttt >> kNumMoveBits; } \
*(prob) = (CLzmaProb)ttt; \
RC_NORM(p) \
@@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
#else
-#define RC_BIT(p, prob, symbol) { \
+#define RC_BIT(p, prob, bit) { \
UInt32 mask; \
RC_BIT_PRE(p, prob) \
- mask = 0 - (UInt32)symbol; \
+ mask = 0 - (UInt32)bit; \
range &= mask; \
mask &= newBound; \
range -= mask; \
(p)->low += mask; \
- mask = (UInt32)symbol - 1; \
+ mask = (UInt32)bit - 1; \
range += newBound & mask; \
mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
mask += ((1 << kNumMoveBits) - 1); \
@@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p, CLzmaProb *prob)
p->range = range;
}
-static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
+static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
{
UInt32 range = p->range;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
UInt32 ttt, newBound;
- // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
- CLzmaProb *prob = probs + (symbol >> 8);
- UInt32 bit = (symbol >> 7) & 1;
- symbol <<= 1;
+ // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
+ CLzmaProb *prob = probs + (sym >> 8);
+ UInt32 bit = (sym >> 7) & 1;
+ sym <<= 1;
RC_BIT(p, prob, bit);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
p->range = range;
}
-static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol, UInt32 matchByte)
+static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 sym, UInt32 matchByte)
{
UInt32 range = p->range;
UInt32 offs = 0x100;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
UInt32 ttt, newBound;
CLzmaProb *prob;
UInt32 bit;
matchByte <<= 1;
- // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol >> 8)), (symbol >> 7) & 1);
- prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
- bit = (symbol >> 7) & 1;
- symbol <<= 1;
- offs &= ~(matchByte ^ symbol);
+ // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >> 8)), (sym >> 7) & 1);
+ prob = probs + (offs + (matchByte & offs) + (sym >> 8));
+ bit = (sym >> 7) & 1;
+ sym <<= 1;
+ offs &= ~(matchByte ^ sym);
RC_BIT(p, prob, bit);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
p->range = range;
}
@@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice *ProbPrices)
}
-#define GET_PRICE(prob, symbol) \
- p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
+#define GET_PRICE(prob, bit) \
+ p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
-#define GET_PRICEa(prob, symbol) \
- ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
+#define GET_PRICEa(prob, bit) \
+ ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
#define GET_PRICE_0(prob) p->ProbPrices[(prob) >> kNumMoveReducingBits]
#define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
@@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice *ProbPrices)
#define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
-static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol, const CProbPrice *ProbPrices)
+static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const CProbPrice *ProbPrices)
{
UInt32 price = 0;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
- unsigned bit = symbol & 1;
- symbol >>= 1;
- price += GET_PRICEa(probs[symbol], bit);
+ unsigned bit = sym & 1;
+ sym >>= 1;
+ price += GET_PRICEa(probs[sym], bit);
}
- while (symbol >= 2);
+ while (sym >= 2);
return price;
}
-static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32 symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
+static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32 sym, UInt32 matchByte, const CProbPrice *ProbPrices)
{
UInt32 price = 0;
UInt32 offs = 0x100;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
matchByte <<= 1;
- price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)], (symbol >> 7) & 1);
- symbol <<= 1;
- offs &= ~(matchByte ^ symbol);
+ price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)], (sym >> 7) & 1);
+ sym <<= 1;
+ offs &= ~(matchByte ^ sym);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
return price;
}
-static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, unsigned numBits, UInt32 symbol)
+static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, unsigned numBits, unsigned sym)
{
UInt32 range = rc->range;
unsigned m = 1;
do
{
UInt32 ttt, newBound;
- unsigned bit = symbol & 1;
+ unsigned bit = sym & 1;
// RangeEnc_EncodeBit(rc, probs + m, bit);
- symbol >>= 1;
+ sym >>= 1;
RC_BIT(rc, probs + m, bit);
m = (m << 1) | bit;
}
@@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
p->high[i] = kProbInitValue;
}
-static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol, unsigned posState)
+static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym, unsigned posState)
{
UInt32 range, ttt, newBound;
CLzmaProb *probs = p->low;
range = rc->range;
RC_BIT_PRE(rc, probs);
- if (symbol >= kLenNumLowSymbols)
+ if (sym >= kLenNumLowSymbols)
{
RC_BIT_1(rc, probs);
probs += kLenNumLowSymbols;
RC_BIT_PRE(rc, probs);
- if (symbol >= kLenNumLowSymbols * 2)
+ if (sym >= kLenNumLowSymbols * 2)
{
RC_BIT_1(rc, probs);
rc->range = range;
- // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol - kLenNumLowSymbols * 2);
- LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
+ // RcTree_Encode(rc, p->high, kLenNumHighBits, sym - kLenNumLowSymbols * 2);
+ LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
return;
}
- symbol -= kLenNumLowSymbols;
+ sym -= kLenNumLowSymbols;
}
- // RcTree_Encode(rc, probs + (posState << kLenNumLowBits), kLenNumLowBits, symbol);
+ // RcTree_Encode(rc, probs + (posState << kLenNumLowBits), kLenNumLowBits, sym);
{
unsigned m;
unsigned bit;
RC_BIT_0(rc, probs);
probs += (posState << (1 + kLenNumLowBits));
- bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
- bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
- bit = symbol & 1; RC_BIT(rc, probs + m, bit);
+ bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
+ bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
+ bit = sym & 1; RC_BIT(rc, probs + m, bit);
rc->range = range;
}
}
@@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs, UInt32 startPrice, UInt32 *price
}
-MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
- CLenPriceEnc *p, unsigned posState,
+MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
+ CLenPriceEnc *p,
+ unsigned numPosStates,
const CLenEnc *enc,
const CProbPrice *ProbPrices)
{
- // int y; for (y = 0; y < 100; y++) {
- UInt32 a;
- unsigned i, numSymbols;
+ UInt32 b;
+
+ {
+ unsigned prob = enc->low[0];
+ UInt32 a, c;
+ unsigned posState;
+ b = GET_PRICEa_1(prob);
+ a = GET_PRICEa_0(prob);
+ c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
+ for (posState = 0; posState < numPosStates; posState++)
+ {
+ UInt32 *prices = p->prices[posState];
+ const CLzmaProb *probs = enc->low + (posState << (1 + kLenNumLowBits));
+ SetPrices_3(probs, a, prices, ProbPrices);
+ SetPrices_3(probs + kLenNumLowSymbols, c, prices + kLenNumLowSymbols, ProbPrices);
+ }
+ }
- UInt32 *prices = p->prices[posState];
+ /*
{
- const CLzmaProb *probs = enc->low + (posState << (1 + kLenNumLowBits));
- SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
+ unsigned i;
+ UInt32 b;
+ a = GET_PRICEa_0(enc->low[0]);
+ for (i = 0; i < kLenNumLowSymbols; i++)
+ p->prices2[i] = a;
a = GET_PRICEa_1(enc->low[0]);
- SetPrices_3(probs + kLenNumLowSymbols, a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices + kLenNumLowSymbols, ProbPrices);
+ b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
+ for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
+ p->prices2[i] = b;
a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
}
- numSymbols = p->tableSize;
- p->counters[posState] = numSymbols;
- for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
+ */
+
+ // p->counter = numSymbols;
+ // p->counter = 64;
+
{
- prices[i] = a +
- // RcTree_GetPrice(enc->high, kLenNumHighBits, i - kLenNumLowSymbols * 2, ProbPrices);
- LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2, ProbPrices);
- /*
- unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
- UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1, sym, ProbPrices);
- UInt32 prob = enc->high[(1 << 7) + sym];
- prices[i ] = price + GET_PRICEa_0(prob);
- prices[i + 1] = price + GET_PRICEa_1(prob);
- */
- }
- // }
-}
+ unsigned i = p->tableSize;
-static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
- const CLenEnc *enc,
- const CProbPrice *ProbPrices)
-{
- unsigned posState;
- for (posState = 0; posState < numPosStates; posState++)
- LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
-}
+ if (i > kLenNumLowSymbols * 2)
+ {
+ const CLzmaProb *probs = enc->high;
+ UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
+ i -= kLenNumLowSymbols * 2 - 1;
+ i >>= 1;
+ b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
+ do
+ {
+ /*
+ p->prices2[i] = a +
+ // RcTree_GetPrice(enc->high, kLenNumHighBits, i - kLenNumLowSymbols * 2, ProbPrices);
+ LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
+ */
+ // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1, sym, ProbPrices);
+ unsigned sym = --i + (1 << (kLenNumHighBits - 1));
+ UInt32 price = b;
+ do
+ {
+ unsigned bit = sym & 1;
+ sym >>= 1;
+ price += GET_PRICEa(probs[sym], bit);
+ }
+ while (sym >= 2);
+
+ {
+ unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits - 1))];
+ prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
+ prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
+ }
+ }
+ while (i);
+ {
+ unsigned posState;
+ size_t num = (p->tableSize - kLenNumLowSymbols * 2) * sizeof(p->prices[0][0]);
+ for (posState = 1; posState < numPosStates; posState++)
+ memcpy(p->prices[posState] + kLenNumLowSymbols * 2, p->prices[0] + kLenNumLowSymbols * 2, num);
+ }
+ }
+ }
+}
/*
#ifdef SHOW_STAT
@@ -914,21 +973,21 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
printf("\n MovePos %u", num);
#endif
*/
-
+
#define MOVE_POS(p, num) { \
p->additionalOffset += (num); \
- p->matchFinder.Skip(p->matchFinderObj, (num)); }
+ p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
{
unsigned numPairs;
-
+
p->additionalOffset++;
p->numAvail = p->matchFinder.GetNumAvailableBytes(p->matchFinderObj);
numPairs = p->matchFinder.GetMatches(p->matchFinderObj, p->matches);
*numPairsRes = numPairs;
-
+
#ifdef SHOW_STAT
printf("\n i = %u numPairs = %u ", g_STAT_OFFSET, numPairs / 2);
g_STAT_OFFSET++;
@@ -938,7 +997,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
printf("%2u %6u | ", p->matches[i], p->matches[i + 1]);
}
#endif
-
+
if (numPairs == 0)
return 0;
{
@@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
const Byte *p2 = p1 + len;
ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
const Byte *lim = p1 + numAvail;
- for (; p2 != lim && *p2 == p2[dif]; p2++);
+ for (; p2 != lim && *p2 == p2[dif]; p2++)
+ {}
return (unsigned)(p2 - p1);
}
}
@@ -976,8 +1036,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
+ GET_PRICE_1(p->isRep0Long[state][posState])) \
+ GET_PRICE_1(p->isRep[state]) \
+ GET_PRICE_0(p->isRepG0[state])
-
+MY_FORCE_INLINE
static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex, size_t state, size_t posState)
{
UInt32 price;
@@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p, unsigned cur)
for (;;)
{
UInt32 dist = p->opt[cur].dist;
- UInt32 len = p->opt[cur].len;
- UInt32 extra = p->opt[cur].extra;
+ unsigned len = (unsigned)p->opt[cur].len;
+ unsigned extra = (unsigned)p->opt[cur].extra;
cur -= len;
if (extra)
{
wr--;
- p->opt[wr].len = len;
+ p->opt[wr].len = (UInt32)len;
cur -= extra;
len = extra;
if (extra == 1)
@@ -1042,10 +1102,10 @@ static unsigned Backward(CLzmaEnc *p, unsigned cur)
p->optCur = wr;
return len;
}
-
+
wr--;
p->opt[wr].dist = dist;
- p->opt[wr].len = len;
+ p->opt[wr].len = (UInt32)len;
}
}
@@ -1068,9 +1128,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
UInt32 matchPrice, repMatchPrice;
const Byte *data;
Byte curByte, matchByte;
-
+
p->optCur = p->optEnd = 0;
-
+
if (p->additionalOffset == 0)
mainLen = ReadMatchDistances(p, &numPairs);
else
@@ -1078,7 +1138,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
mainLen = p->longestMatchLen;
numPairs = p->numPairs;
}
-
+
numAvail = p->numAvail;
if (numAvail < 2)
{
@@ -1087,10 +1147,10 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
if (numAvail > LZMA_MATCH_LEN_MAX)
numAvail = LZMA_MATCH_LEN_MAX;
-
+
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
repMaxIndex = 0;
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned len;
@@ -1102,43 +1162,48 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
repLens[i] = 0;
continue;
}
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
repLens[i] = len;
if (len > repLens[repMaxIndex])
repMaxIndex = i;
}
-
+
if (repLens[repMaxIndex] >= p->numFastBytes)
{
unsigned len;
- p->backRes = repMaxIndex;
+ p->backRes = (UInt32)repMaxIndex;
len = repLens[repMaxIndex];
MOVE_POS(p, len - 1)
return len;
}
-
+
matches = p->matches;
-
+
if (mainLen >= p->numFastBytes)
{
p->backRes = matches[(size_t)numPairs - 1] + LZMA_NUM_REPS;
MOVE_POS(p, mainLen - 1)
return mainLen;
}
-
+
curByte = *data;
matchByte = *(data - reps[0]);
-
- if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] < 2)
+
+ last = repLens[repMaxIndex];
+ if (last <= mainLen)
+ last = mainLen;
+
+ if (last < 2 && curByte != matchByte)
{
p->backRes = MARK_LIT;
return 1;
}
-
+
p->opt[0].state = (CState)p->state;
-
+
posState = (position & p->pbMask);
-
+
{
const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
p->opt[1].price = GET_PRICE_0(p->isMatch[p->state][posState]) +
@@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
LitEnc_Matched_GetPrice(probs, curByte, matchByte, p->ProbPrices) :
LitEnc_GetPrice(probs, curByte, p->ProbPrices));
}
-
+
MakeAs_Lit(&p->opt[1]);
-
+
matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
-
- if (matchByte == curByte)
+
+ // 18.06
+ if (matchByte == curByte && repLens[0] == 0)
{
UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, p->state, posState);
if (shortRepPrice < p->opt[1].price)
@@ -1160,32 +1226,22 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
p->opt[1].price = shortRepPrice;
MakeAs_ShortRep(&p->opt[1]);
}
+ if (last < 2)
+ {
+ p->backRes = p->opt[1].dist;
+ return 1;
+ }
}
-
- last = (mainLen >= repLens[repMaxIndex] ? mainLen : repLens[repMaxIndex]);
-
- if (last < 2)
- {
- p->backRes = p->opt[1].dist;
- return 1;
- }
-
+
p->opt[1].len = 1;
-
+
p->opt[0].reps[0] = reps[0];
p->opt[0].reps[1] = reps[1];
p->opt[0].reps[2] = reps[2];
p->opt[0].reps[3] = reps[3];
-
- {
- unsigned len = last;
- do
- p->opt[len--].price = kInfinityPrice;
- while (len >= 2);
- }
// ---------- REP ----------
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned repLen = repLens[i];
@@ -1195,58 +1251,61 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
do
{
- UInt32 price2 = price + p->repLenEnc.prices[posState][(size_t)repLen - 2];
+ UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState, repLen);
COptimal *opt = &p->opt[repLen];
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = repLen;
- opt->dist = i;
+ opt->len = (UInt32)repLen;
+ opt->dist = (UInt32)i;
opt->extra = 0;
}
}
while (--repLen >= 2);
}
-
-
+
+
// ---------- MATCH ----------
{
- unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
+ unsigned len = repLens[0] + 1;
if (len <= mainLen)
{
unsigned offs = 0;
UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[p->state]);
- while (len > matches[offs])
- offs += 2;
-
+ if (len < 2)
+ len = 2;
+ else
+ while (len > matches[offs])
+ offs += 2;
+
for (; ; len++)
{
COptimal *opt;
UInt32 dist = matches[(size_t)offs + 1];
- UInt32 price2 = normalMatchPrice + p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
+ UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc, posState, len);
unsigned lenToPosState = GetLenToPosState(len);
-
+
if (dist < kNumFullDistances)
- price2 += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
+ price += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
else
{
unsigned slot;
GetPosSlot2(dist, slot);
- price2 += p->alignPrices[dist & kAlignMask];
- price2 += p->posSlotPrices[lenToPosState][slot];
+ price += p->alignPrices[dist & kAlignMask];
+ price += p->posSlotPrices[lenToPosState][slot];
}
-
+
opt = &p->opt[len];
-
- if (price2 < opt->price)
+
+ if (price < opt->price)
{
- opt->price = price2;
- opt->len = len;
+ opt->price = price;
+ opt->len = (UInt32)len;
opt->dist = dist + LZMA_NUM_REPS;
opt->extra = 0;
}
-
+
if (len == matches[offs])
{
offs += 2;
@@ -1256,7 +1315,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
-
+
cur = 0;
@@ -1272,37 +1331,73 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
-
+
// ---------- Optimal Parsing ----------
for (;;)
{
- UInt32 numAvail, numAvailFull;
+ unsigned numAvail;
+ UInt32 numAvailFull;
unsigned newLen, numPairs, prev, state, posState, startLen;
- UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
- Bool nextIsLit;
+ UInt32 litPrice, matchPrice, repMatchPrice;
+ BoolInt nextIsLit;
Byte curByte, matchByte;
const Byte *data;
COptimal *curOpt, *nextOpt;
if (++cur == last)
- return Backward(p, cur);
+ break;
+
+ // 18.06
+ if (cur >= kNumOpts - 64)
+ {
+ unsigned j, best;
+ UInt32 price = p->opt[cur].price;
+ best = cur;
+ for (j = cur + 1; j <= last; j++)
+ {
+ UInt32 price2 = p->opt[j].price;
+ if (price >= price2)
+ {
+ price = price2;
+ best = j;
+ }
+ }
+ {
+ unsigned delta = best - cur;
+ if (delta != 0)
+ {
+ MOVE_POS(p, delta);
+ }
+ }
+ cur = best;
+ break;
+ }
newLen = ReadMatchDistances(p, &numPairs);
-
+
if (newLen >= p->numFastBytes)
{
p->numPairs = numPairs;
p->longestMatchLen = newLen;
- return Backward(p, cur);
+ break;
}
-
+
curOpt = &p->opt[cur];
+
+ position++;
+
+ // we need that check here, if skip_items in p->opt are possible
+ /*
+ if (curOpt->price >= kInfinityPrice)
+ continue;
+ */
+
prev = cur - curOpt->len;
-
+
if (curOpt->len == 1)
{
- state = p->opt[prev].state;
+ state = (unsigned)p->opt[prev].state;
if (IsShortRep(curOpt))
state = kShortRepNextStates[state];
else
@@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (curOpt->extra)
{
- prev -= curOpt->extra;
+ prev -= (unsigned)curOpt->extra;
state = kState_RepAfterLit;
if (curOpt->extra == 1)
- state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit : kState_MatchAfterLit;
+ state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit : kState_MatchAfterLit);
}
else
{
- state = p->opt[prev].state;
+ state = (unsigned)p->opt[prev].state;
if (dist < LZMA_NUM_REPS)
state = kRepNextStates[state];
else
@@ -1368,7 +1463,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
reps[3] = prevOpt->reps[2];
}
}
-
+
curOpt->state = (CState)state;
curOpt->reps[0] = reps[0];
curOpt->reps[1] = reps[1];
@@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
curByte = *data;
matchByte = *(data - reps[0]);
- position++;
posState = (position & p->pbMask);
/*
@@ -1391,19 +1485,31 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
< MATCH [ : LIT : REP_0 ]
*/
- curPrice = curOpt->price;
- litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
+ {
+ UInt32 curPrice = curOpt->price;
+ unsigned prob = p->isMatch[state][posState];
+ matchPrice = curPrice + GET_PRICE_1(prob);
+ litPrice = curPrice + GET_PRICE_0(prob);
+ }
nextOpt = &p->opt[(size_t)cur + 1];
nextIsLit = False;
- // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
+ // here we can allow skip_items in p->opt, if we don't check (nextOpt->price < kInfinityPrice)
+ // 18.new.06
+ if ((nextOpt->price < kInfinityPrice
+ // && !IsLitState(state)
+ && matchByte == curByte)
+ || litPrice > nextOpt->price
+ )
+ litPrice = 0;
+ else
{
const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
litPrice += (!IsLitState(state) ?
LitEnc_Matched_GetPrice(probs, curByte, matchByte, p->ProbPrices) :
LitEnc_GetPrice(probs, curByte, p->ProbPrices));
-
+
if (litPrice < nextOpt->price)
{
nextOpt->price = litPrice;
@@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
- matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
-
+
+ numAvailFull = p->numAvail;
+ {
+ unsigned temp = kNumOpts - 1 - cur;
+ if (numAvailFull > temp)
+ numAvailFull = (UInt32)temp;
+ }
+
+ // 18.06
// ---------- SHORT_REP ----------
- // if (IsLitState(state)) // 18.new
+ if (IsLitState(state)) // 18.new
if (matchByte == curByte)
- // if (repMatchPrice < nextOpt->price) // 18.new
- if (nextOpt->len < 2
+ if (repMatchPrice < nextOpt->price) // 18.new
+ // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
+ if (
+ // nextOpt->price >= kInfinityPrice ||
+ nextOpt->len < 2 // we can check nextOpt->len, if skip items are not allowed in p->opt
|| (nextOpt->dist != 0
- && nextOpt->extra <= 1 // 17.old
- ))
+ // && nextOpt->extra <= 1 // 17.old
+ )
+ )
{
UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state, posState);
- if (shortRepPrice <= nextOpt->price) // 17.old
- // if (shortRepPrice < nextOpt->price) // 18.new
+ // if (shortRepPrice <= nextOpt->price) // 17.old
+ if (shortRepPrice < nextOpt->price) // 18.new
{
nextOpt->price = shortRepPrice;
nextOpt->len = 1;
@@ -1435,13 +1552,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
nextIsLit = False;
}
}
-
- numAvailFull = p->numAvail;
- {
- UInt32 temp = kNumOpts - 1 - cur;
- if (numAvailFull > temp)
- numAvailFull = temp;
- }
if (numAvailFull < 2)
continue;
@@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
// ---------- LIT : REP_0 ----------
- if (
- // litPrice != 0 && // 18.new
- !nextIsLit
+ if (!nextIsLit
+ && litPrice != 0 // 18.new
&& matchByte != curByte
&& numAvailFull > 2)
{
@@ -1465,32 +1574,32 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (limit > numAvailFull)
limit = numAvailFull;
for (len = 3; len < limit && data[len] == data2[len]; len++)
- {
- }
-
+ {}
+
{
unsigned state2 = kLiteralNextStates[state];
unsigned posState2 = (position + 1) & p->pbMask;
UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
{
unsigned offset = cur + len;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
-
+
+ if (last < offset)
+ last = offset;
+
// do
{
UInt32 price2;
COptimal *opt;
len--;
// price2 = price + GetPrice_Len_Rep_0(p, len, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len;
+ opt->len = (UInt32)len;
opt->dist = 0;
opt->extra = 1;
}
@@ -1500,8 +1609,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
-
+
startLen = 2; /* speed optimization */
+
{
// ---------- REP ----------
unsigned repIndex = 0; // 17.old
@@ -1513,31 +1623,35 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
const Byte *data2 = data - reps[repIndex];
if (data[0] != data2[0] || data[1] != data2[1])
continue;
-
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
-
+
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
+
// if (len < startLen) continue; // 18.new: speed optimization
- while (last < cur + len)
- p->opt[++last].price = kInfinityPrice;
+ {
+ unsigned offset = cur + len;
+ if (last < offset)
+ last = offset;
+ }
{
unsigned len2 = len;
price = repMatchPrice + GetPrice_PureRep(p, repIndex, state, posState);
do
{
- UInt32 price2 = price + p->repLenEnc.prices[posState][(size_t)len2 - 2];
+ UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState, len2);
COptimal *opt = &p->opt[cur + len2];
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
- opt->dist = repIndex;
+ opt->len = (UInt32)len2;
+ opt->dist = (UInt32)repIndex;
opt->extra = 0;
}
}
while (--len2 >= 2);
}
-
+
if (repIndex == 0) startLen = len + 1; // 17.old
// startLen = len + 1; // 18.new
@@ -1550,51 +1664,59 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
unsigned limit = len2 + p->numFastBytes;
if (limit > numAvailFull)
limit = numAvailFull;
-
- for (; len2 < limit && data[len2] == data2[len2]; len2++);
-
- len2 -= len;
- if (len2 >= 3)
+
+ len2 += 2;
+ if (len2 <= limit)
+ if (data[len2 - 2] == data2[len2 - 2])
+ if (data[len2 - 1] == data2[len2 - 1])
{
unsigned state2 = kRepNextStates[state];
unsigned posState2 = (position + len) & p->pbMask;
- price +=
- p->repLenEnc.prices[posState][(size_t)len - 2]
+ price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
+ GET_PRICE_0(p->isMatch[state2][posState2])
+ LitEnc_Matched_GetPrice(LIT_PROBS(position + len, data[(size_t)len - 1]),
data[len], data2[len], p->ProbPrices);
-
+
// state2 = kLiteralNextStates[state2];
state2 = kState_LitAfterRep;
posState2 = (posState2 + 1) & p->pbMask;
price += GetPrice_Rep_0(p, state2, posState2);
+
+ for (; len2 < limit && data[len2] == data2[len2]; len2++)
+ {}
+
+ len2 -= len;
+ // if (len2 >= 3)
+ {
{
unsigned offset = cur + len + len2;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
- unsigned price2;
+ UInt32 price2;
COptimal *opt;
len2--;
// price2 = price + GetPrice_Len_Rep_0(p, len2, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len2 - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len2);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
+ opt->len = (UInt32)len2;
opt->extra = (CExtra)(len + 1);
- opt->dist = repIndex;
+ opt->dist = (UInt32)repIndex;
}
}
// while (len2 >= 3);
}
}
+ }
}
}
}
@@ -1606,49 +1728,57 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
{
newLen = numAvail;
for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
- matches[numPairs] = newLen;
+ matches[numPairs] = (UInt32)newLen;
numPairs += 2;
}
-
+
+ // startLen = 2; /* speed optimization */
+
if (newLen >= startLen)
{
UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[state]);
UInt32 dist;
unsigned offs, posSlot, len;
- while (last < cur + newLen)
- p->opt[++last].price = kInfinityPrice;
+
+ {
+ unsigned offset = cur + newLen;
+ if (last < offset)
+ last = offset;
+ }
offs = 0;
while (startLen > matches[offs])
offs += 2;
dist = matches[(size_t)offs + 1];
-
+
// if (dist >= kNumFullDistances)
GetPosSlot2(dist, posSlot);
-
+
for (len = /*2*/ startLen; ; len++)
{
- UInt32 price = normalMatchPrice + p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
+ UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc, posState, len);
{
COptimal *opt;
- unsigned lenToPosState = len - 2; lenToPosState = GetLenToPosState2(lenToPosState);
+ unsigned lenNorm = len - 2;
+ lenNorm = GetLenToPosState2(lenNorm);
if (dist < kNumFullDistances)
- price += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
+ price += p->distancesPrices[lenNorm][dist & (kNumFullDistances - 1)];
else
- price += p->posSlotPrices[lenToPosState][posSlot] + p->alignPrices[dist & kAlignMask];
-
+ price += p->posSlotPrices[lenNorm][posSlot] + p->alignPrices[dist & kAlignMask];
+
opt = &p->opt[cur + len];
if (price < opt->price)
{
opt->price = price;
- opt->len = len;
+ opt->len = (UInt32)len;
opt->dist = dist + LZMA_NUM_REPS;
opt->extra = 0;
}
}
- if (/*_maxMode && */ len == matches[offs])
+ if (len == matches[offs])
{
+ // if (p->_maxMode) {
// MATCH : LIT : REP_0
const Byte *data2 = data - dist - 1;
@@ -1656,12 +1786,18 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
unsigned limit = len2 + p->numFastBytes;
if (limit > numAvailFull)
limit = numAvailFull;
-
- for (; len2 < limit && data[len2] == data2[len2]; len2++);
-
+
+ len2 += 2;
+ if (len2 <= limit)
+ if (data[len2 - 2] == data2[len2 - 2])
+ if (data[len2 - 1] == data2[len2 - 1])
+ {
+ for (; len2 < limit && data[len2] == data2[len2]; len2++)
+ {}
+
len2 -= len;
-
- if (len2 >= 3)
+
+ // if (len2 >= 3)
{
unsigned state2 = kMatchNextStates[state];
unsigned posState2 = (position + len) & p->pbMask;
@@ -1677,28 +1813,31 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price += GetPrice_Rep_0(p, state2, posState2);
offset = cur + len + len2;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
UInt32 price2;
COptimal *opt;
len2--;
// price2 = price + GetPrice_Len_Rep_0(p, len2, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len2 - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len2);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
+ opt->len = (UInt32)len2;
opt->extra = (CExtra)(len + 1);
opt->dist = dist + LZMA_NUM_REPS;
}
}
// while (len2 >= 3);
}
-
+
+ }
+
offs += 2;
if (offs == numPairs)
break;
@@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
+
+ do
+ p->opt[last].price = kInfinityPrice;
+ while (--last);
+
+ return Backward(p, cur);
}
@@ -1735,21 +1880,23 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
p->backRes = MARK_LIT;
if (numAvail < 2)
return 1;
+ // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
if (numAvail > LZMA_MATCH_LEN_MAX)
numAvail = LZMA_MATCH_LEN_MAX;
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
repLen = repIndex = 0;
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned len;
const Byte *data2 = data - p->reps[i];
if (data[0] != data2[0] || data[1] != data2[1])
continue;
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
if (len >= p->numFastBytes)
{
- p->backRes = i;
+ p->backRes = (UInt32)i;
MOVE_POS(p, len - 1)
return len;
}
@@ -1768,7 +1915,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
}
mainDist = 0; /* for GCC */
-
+
if (mainLen >= 2)
{
mainDist = p->matches[(size_t)numPairs - 1];
@@ -1793,18 +1940,18 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
|| (repLen + 2 >= mainLen && mainDist >= (1 << 9))
|| (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
{
- p->backRes = repIndex;
+ p->backRes = (UInt32)repIndex;
MOVE_POS(p, repLen - 1)
return repLen;
}
-
+
if (mainLen < 2 || numAvail <= 2)
return 1;
{
unsigned len1 = ReadMatchDistances(p, &p->numPairs);
p->longestMatchLen = len1;
-
+
if (len1 >= 2)
{
UInt32 newDist = p->matches[(size_t)p->numPairs - 1];
@@ -1815,9 +1962,9 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
return 1;
}
}
-
+
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
-
+
for (i = 0; i < LZMA_NUM_REPS; i++)
{
unsigned len, limit;
@@ -1833,7 +1980,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
break;
}
}
-
+
p->backRes = mainDist + LZMA_NUM_REPS;
if (mainLen != 2)
{
@@ -1859,7 +2006,7 @@ static void WriteEndMarker(CLzmaEnc *p, unsigned posState)
RC_BIT_0(&p->rc, prob)
}
p->state = kMatchNextStates[p->state];
-
+
p->rc.range = range;
LenEnc_Encode(&p->lenProbs, &p->rc, 0, posState);
range = p->rc.range;
@@ -1888,7 +2035,7 @@ static void WriteEndMarker(CLzmaEnc *p, unsigned posState)
}
while (--numBits);
}
-
+
{
// RcTree_ReverseEncode(&p->rc, p->posAlignEncoder, kNumAlignBits, kAlignMask);
CLzmaProb *probs = p->posAlignEncoder;
@@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p, UInt32 nowPos)
}
-
-static void FillAlignPrices(CLzmaEnc *p)
+MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
{
unsigned i;
const CProbPrice *ProbPrices = p->ProbPrices;
const CLzmaProb *probs = p->posAlignEncoder;
- p->alignPriceCount = 0;
+ // p->alignPriceCount = 0;
for (i = 0; i < kAlignTableSize / 2; i++)
{
UInt32 price = 0;
- unsigned symbol = i;
+ unsigned sym = i;
unsigned m = 1;
unsigned bit;
UInt32 prob;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
prob = probs[m];
p->alignPrices[i ] = price + GET_PRICEa_0(prob);
p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
@@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
}
-static void FillDistancesPrices(CLzmaEnc *p)
+MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
{
+ // int y; for (y = 0; y < 100; y++) {
+
UInt32 tempPrices[kNumFullDistances];
- unsigned i, lenToPosState;
+ unsigned i, lps;
const CProbPrice *ProbPrices = p->ProbPrices;
p->matchPriceCount = 0;
- for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
+ for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
{
unsigned posSlot = GetPosSlot1(i);
- unsigned footerBits = ((posSlot >> 1) - 1);
+ unsigned footerBits = (posSlot >> 1) - 1;
unsigned base = ((2 | (posSlot & 1)) << footerBits);
+ const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
// tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base, footerBits, i - base, p->ProbPrices);
-
- const CLzmaProb *probs = p->posEncoders + base;
UInt32 price = 0;
unsigned m = 1;
- unsigned symbol = i - base;
+ unsigned sym = i;
+ unsigned offset = (unsigned)1 << footerBits;
+ base += i;
+
+ if (footerBits)
do
{
- unsigned bit = symbol & 1;
- symbol >>= 1;
+ unsigned bit = sym & 1;
+ sym >>= 1;
price += GET_PRICEa(probs[m], bit);
m = (m << 1) + bit;
}
while (--footerBits);
- tempPrices[i] = price;
+
+ {
+ unsigned prob = probs[m];
+ tempPrices[base ] = price + GET_PRICEa_0(prob);
+ tempPrices[base + offset] = price + GET_PRICEa_1(prob);
+ }
}
- for (lenToPosState = 0; lenToPosState < kNumLenToPosStates; lenToPosState++)
+ for (lps = 0; lps < kNumLenToPosStates; lps++)
{
- unsigned posSlot;
- const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
- UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
- unsigned distTableSize = p->distTableSize;
- const CLzmaProb *probs = encoder;
- for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
+ unsigned slot;
+ unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
+ UInt32 *posSlotPrices = p->posSlotPrices[lps];
+ const CLzmaProb *probs = p->posSlotEncoder[lps];
+
+ for (slot = 0; slot < distTableSize2; slot++)
{
- // posSlotPrices[posSlot] = RcTree_GetPrice(encoder, kNumPosSlotBits, posSlot, p->ProbPrices);
- UInt32 price = 0;
+ // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits, slot, p->ProbPrices);
+ UInt32 price;
unsigned bit;
- unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
- UInt32 prob;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
- posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
- posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
+ unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
+ unsigned prob;
+ bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
+ posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
+ posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
}
- for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
- posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
{
- UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
+ UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
+ for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
{
- distancesPrices[0] = posSlotPrices[0];
- distancesPrices[1] = posSlotPrices[1];
- distancesPrices[2] = posSlotPrices[2];
- distancesPrices[3] = posSlotPrices[3];
+ posSlotPrices[(size_t)slot * 2 ] += delta;
+ posSlotPrices[(size_t)slot * 2 + 1] += delta;
+ delta += ((UInt32)1 << kNumBitPriceShiftBits);
}
+ }
+
+ {
+ UInt32 *dp = p->distancesPrices[lps];
+
+ dp[0] = posSlotPrices[0];
+ dp[1] = posSlotPrices[1];
+ dp[2] = posSlotPrices[2];
+ dp[3] = posSlotPrices[3];
+
for (i = 4; i < kNumFullDistances; i += 2)
{
UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
- distancesPrices[i ] = slotPrice + tempPrices[i];
- distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
+ dp[i ] = slotPrice + tempPrices[i];
+ dp[i + 1] = slotPrice + tempPrices[i + 1];
}
}
}
+ // }
}
@@ -2037,7 +2202,7 @@ void LzmaEnc_Construct(CLzmaEnc *p)
{
RangeEnc_Construct(&p->rc);
MatchFinder_Construct(&p->matchFinderBase);
-
+
#ifndef _7ZIP_ST
MatchFinderMt_Construct(&p->matchFinderMt);
p->matchFinderMt.MatchFinder = &p->matchFinderBase;
@@ -2081,7 +2246,7 @@ void LzmaEnc_Destruct(CLzmaEnc *p, ISzAllocPtr alloc, ISzAllocPtr allocBig)
#ifndef _7ZIP_ST
MatchFinderMt_Destruct(&p->matchFinderMt, allocBig);
#endif
-
+
MatchFinder_Free(&p->matchFinderBase, allocBig);
LzmaEnc_FreeLits(p, alloc);
RangeEnc_Free(&p->rc, alloc);
@@ -2126,14 +2291,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
}
if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) != 0)
-
+
for (;;)
{
UInt32 dist;
unsigned len, posState;
UInt32 range, ttt, newBound;
CLzmaProb *probs;
-
+
if (p->fastMode)
len = GetOptimumFast(p);
else
@@ -2153,9 +2318,9 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
posState = (unsigned)nowPos32 & p->pbMask;
range = p->rc.range;
probs = &p->isMatch[p->state][posState];
-
+
RC_BIT_PRE(&p->rc, probs)
-
+
dist = p->backRes;
#ifdef SHOW_STAT2
@@ -2185,7 +2350,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
RC_BIT_1(&p->rc, probs);
probs = &p->isRep[p->state];
RC_BIT_PRE(&p->rc, probs)
-
+
if (dist < LZMA_NUM_REPS)
{
RC_BIT_1(&p->rc, probs);
@@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (len != 1)
{
LenEnc_Encode(&p->repLenProbs, &p->rc, len - LZMA_MATCH_LEN_MIN, posState);
- if (!p->fastMode)
- if (--p->repLenEnc.counters[posState] == 0)
- LenPriceEnc_UpdateTable(&p->repLenEnc, posState, &p->repLenProbs, p->ProbPrices);
-
+ --p->repLenEncCounter;
p->state = kRepNextStates[p->state];
}
}
@@ -2260,34 +2422,32 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
p->state = kMatchNextStates[p->state];
LenEnc_Encode(&p->lenProbs, &p->rc, len - LZMA_MATCH_LEN_MIN, posState);
- if (!p->fastMode)
- if (--p->lenEnc.counters[posState] == 0)
- LenPriceEnc_UpdateTable(&p->lenEnc, posState, &p->lenProbs, p->ProbPrices);
+ // --p->lenEnc.counter;
dist -= LZMA_NUM_REPS;
p->reps[3] = p->reps[2];
p->reps[2] = p->reps[1];
p->reps[1] = p->reps[0];
p->reps[0] = dist + 1;
-
+
p->matchPriceCount++;
GetPosSlot(dist, posSlot);
// RcTree_Encode_PosSlot(&p->rc, p->posSlotEncoder[GetLenToPosState(len)], posSlot);
{
- UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
+ UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
range = p->rc.range;
probs = p->posSlotEncoder[GetLenToPosState(len)];
do
{
- CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
- UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
- symbol <<= 1;
+ CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
+ UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
+ sym <<= 1;
RC_BIT(&p->rc, prob, bit);
}
- while (symbol < (1 << kNumPosSlotBits * 2));
+ while (sym < (1 << kNumPosSlotBits * 2));
p->rc.range = range;
}
-
+
if (dist >= kStartPosModelIndex)
{
unsigned footerBits = ((posSlot >> 1) - 1);
@@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (dist < kNumFullDistances)
{
unsigned base = ((2 | (posSlot & 1)) << footerBits);
- RcTree_ReverseEncode(&p->rc, p->posEncoders + base, footerBits, dist - base);
+ RcTree_ReverseEncode(&p->rc, p->posEncoders + base, footerBits, (unsigned)(dist /* - base */));
}
else
{
@@ -2331,32 +2491,49 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
bit = dist & 1; dist >>= 1; RC_BIT(&p->rc, p->posAlignEncoder + m, bit); m = (m << 1) + bit;
bit = dist & 1; RC_BIT(&p->rc, p->posAlignEncoder + m, bit);
p->rc.range = range;
- p->alignPriceCount++;
+ // p->alignPriceCount++;
}
}
}
}
}
- nowPos32 += len;
+ nowPos32 += (UInt32)len;
p->additionalOffset -= len;
-
+
if (p->additionalOffset == 0)
{
UInt32 processed;
if (!p->fastMode)
{
- if (p->matchPriceCount >= (1 << 7))
+ /*
+ if (p->alignPriceCount >= 16) // kAlignTableSize
+ FillAlignPrices(p);
+ if (p->matchPriceCount >= 128)
FillDistancesPrices(p);
- if (p->alignPriceCount >= kAlignTableSize)
+ if (p->lenEnc.counter <= 0)
+ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
+ */
+ if (p->matchPriceCount >= 64)
+ {
FillAlignPrices(p);
+ // { int y; for (y = 0; y < 100; y++) {
+ FillDistancesPrices(p);
+ // }}
+ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
+ }
+ if (p->repLenEncCounter <= 0)
+ {
+ p->repLenEncCounter = REP_LEN_COUNT;
+ LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs, p->ProbPrices);
+ }
}
-
+
if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
break;
processed = nowPos32 - startPos32;
-
+
if (maxPackSize)
{
if (processed + kNumOpts + 300 >= maxUnpackSize
@@ -2430,7 +2607,7 @@ static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32 keepWindowSize, ISzAllocPtr alloc,
p->matchFinderObj = &p->matchFinderBase;
MatchFinder_CreateVTable(&p->matchFinderBase, &p->matchFinder);
}
-
+
return SZ_OK;
}
@@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
p->optEnd = 0;
p->optCur = 0;
+
+ {
+ for (i = 0; i < kNumOpts; i++)
+ p->opt[i].price = kInfinityPrice;
+ }
+
p->additionalOffset = 0;
p->pbMask = (1 << p->pb) - 1;
p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
}
+
void LzmaEnc_InitPrices(CLzmaEnc *p)
{
if (!p->fastMode)
@@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
p->lenEnc.tableSize =
p->repLenEnc.tableSize =
p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
+
+ p->repLenEncCounter = REP_LEN_COUNT;
+
LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs, p->ProbPrices);
}
@@ -2583,7 +2770,7 @@ typedef struct
ISeqOutStream vt;
Byte *data;
SizeT rem;
- Bool overflow;
+ BoolInt overflow;
} CLzmaEnc_SeqOutStreamBuf;
static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void *data, size_t size)
@@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle pp)
}
-SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
+SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32 *unpackSize)
{
CLzmaEnc *p = (CLzmaEnc *)pp;
@@ -2644,7 +2831,7 @@ SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
return SZ_ERROR_OUTPUT_EOF;
res = LzmaEnc_CodeOneBlock(p, desiredPackSize, *unpackSize);
-
+
*unpackSize = (UInt32)(p->nowPos64 - nowPos64);
*destLen -= outStream.rem;
if (outStream.overflow)
@@ -2679,7 +2866,7 @@ static SRes LzmaEnc_Encode2(CLzmaEnc *p, ICompressProgress *progress)
}
}
}
-
+
LzmaEnc_Finish(p);
/*
@@ -2751,7 +2938,7 @@ SRes LzmaEnc_MemEncode(CLzmaEncHandle pp, Byte *dest, SizeT *destLen, const Byte
p->rc.outStream = &outStream.vt;
res = LzmaEnc_MemPrepare(pp, src, srcLen, 0, alloc, allocBig);
-
+
if (res == SZ_OK)
{
res = LzmaEnc_Encode2(p, progress);
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
index d7426d38a4..0f9560c7a6 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
@@ -1,13 +1,35 @@
HISTORY of the LZMA SDK
-----------------------
+19.00 2019-02-21
+-------------------------
+- Encryption strength for 7z archives was increased:
+ the size of random initialization vector was increased from 64-bit to 128-bit,
+ and the pseudo-random number generator was improved.
+- The bug in 7zIn.c code was fixed.
+
+
+18.06 2018-12-30
+-------------------------
+- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
+ and there are minor changes in compression ratio.
+- Some bugs were fixed.
+- The bug in 7-Zip 18.02-18.05 was fixed:
+ There was memory leak in multithreading xz decoder - XzDecMt_Decode(),
+ if xz stream contains only one block.
+- The changes for MSVS compiler makefiles:
+ - the makefiles now use "PLATFORM" macroname with values (x64, x86, arm64)
+ instead of "CPU" macroname with values (AMD64, ARM64).
+ - the makefiles by default now use static version of the run-time library.
+
+
18.05 2018-04-30
-------------------------
-- The speed for LZMA/LZMA2 compressing was increased
- by 8% for fastest/fast compression levels and
+- The speed for LZMA/LZMA2 compressing was increased
+ by 8% for fastest/fast compression levels and
by 3% for normal/maximum compression levels.
- Previous versions of 7-Zip could work incorrectly in "Large memory pages" mode in
- Windows 10 because of some BUG with "Large Pages" in Windows 10.
+ Windows 10 because of some BUG with "Large Pages" in Windows 10.
Now 7-Zip doesn't use "Large Pages" on Windows 10 up to revision 1709 (16299).
- The BUG was fixed in Lzma2Enc.c
Lzma2Enc_Encode2() function worked incorretly,
@@ -16,7 +38,7 @@ HISTORY of the LZMA SDK
18.03 beta 2018-03-04
-------------------------
-- Asm\x86\LzmaDecOpt.asm: new optimized LZMA decoder written in asm
+- Asm\x86\LzmaDecOpt.asm: new optimized LZMA decoder written in asm
for x64 with about 30% higher speed than main version of LZMA decoder written in C.
- The speed for single-thread LZMA/LZMA2 decoder written in C was increased by 3%.
- 7-Zip now can use multi-threading for 7z/LZMA2 decoding,
@@ -35,7 +57,7 @@ HISTORY of the LZMA SDK
18.00 beta 2019-01-10
-------------------------
- The BUG in xz encoder was fixed:
- There was memory leak of 16 KB for each file compressed with
+ There was memory leak of 16 KB for each file compressed with
xz compression method, if additional filter was used.
@@ -44,7 +66,7 @@ HISTORY of the LZMA SDK
- Minor speed optimization for LZMA2 (xz and 7z) multi-threading compression.
7-Zip now uses additional memory buffers for multi-block LZMA2 compression.
CPU utilization was slightly improved.
-- 7-zip now creates multi-block xz archives by default. Block size can be
+- 7-zip now creates multi-block xz archives by default. Block size can be
specified with -ms[Size]{m|g} switch.
- xz decoder now can unpack random block from multi-block xz archives.
- 7-Zip command line: @listfile now doesn't work after -- switch.
@@ -55,7 +77,7 @@ HISTORY of the LZMA SDK
17.00 beta 2017-04-29
-------------------------
-- NewHandler.h / NewHandler.cpp:
+- NewHandler.h / NewHandler.cpp:
now it redefines operator new() only for old MSVC compilers (_MSC_VER < 1900).
- C/7zTypes.h : the names of variables in interface structures were changed (vt).
- Some bugs were fixed. 7-Zip could crash in some cases.
@@ -76,51 +98,51 @@ HISTORY of the LZMA SDK
16.02 2016-05-21
-------------------------
- The BUG in 16.00 - 16.01 was fixed:
- Split Handler (SplitHandler.cpp) returned incorrect
+ Split Handler (SplitHandler.cpp) returned incorrect
total size value (kpidSize) for split archives.
16.01 2016-05-19
--------------------------
+-------------------------
- Some internal changes to reduce the number of compiler warnings.
16.00 2016-05-10
--------------------------
+-------------------------
- Some bugs were fixed.
15.12 2015-11-19
--------------------------
+-------------------------
- The BUG in C version of 7z decoder was fixed:
7zDec.c : SzDecodeLzma2()
7z decoder could mistakenly report about decoding error for some 7z archives
that use LZMA2 compression method.
- The probability to get that mistaken decoding error report was about
- one error per 16384 solid blocks for solid blocks larger than 16 KB (compressed size).
+ The probability to get that mistaken decoding error report was about
+ one error per 16384 solid blocks for solid blocks larger than 16 KB (compressed size).
- The BUG (in 9.26-15.11) in C version of 7z decoder was fixed:
7zArcIn.c : SzReadHeader2()
- 7z decoder worked incorrectly for 7z archives that contain
- empty solid blocks, that can be placed to 7z archive, if some file is
+ 7z decoder worked incorrectly for 7z archives that contain
+ empty solid blocks, that can be placed to 7z archive, if some file is
unavailable for reading during archive creation.
15.09 beta 2015-10-16
--------------------------
+-------------------------
- The BUG in LZMA / LZMA2 encoding code was fixed.
The BUG in LzFind.c::MatchFinder_ReadBlock() function.
If input data size is larger than (4 GiB - dictionary_size),
the following code worked incorrectly:
- - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding functions
- for compressing from memory to memory.
+ - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding functions
+ for compressing from memory to memory.
That BUG is not related to LZMA encoder version that works via streams.
- - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly, if
- default value of chunk size (CLzma2EncProps::blockSize) is changed
+ - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly, if
+ default value of chunk size (CLzma2EncProps::blockSize) is changed
to value larger than (4 GiB - dictionary_size).
9.38 beta 2015-01-03
--------------------------
+-------------------------
- The BUG in 9.31-9.37 was fixed:
IArchiveGetRawProps interface was disabled for 7z archives.
- The BUG in 9.26-9.36 was fixed:
@@ -128,10 +150,10 @@ HISTORY of the LZMA SDK
9.36 beta 2014-12-26
--------------------------
+-------------------------
- The BUG in command line version was fixed:
7-Zip created temporary archive in current folder during update archive
- operation, if -w{Path} switch was not specified.
+ operation, if -w{Path} switch was not specified.
The fixed 7-Zip creates temporary archive in folder that contains updated archive.
- The BUG in 9.33-9.35 was fixed:
7-Zip silently ignored file reading errors during 7z or gz archive creation,
@@ -140,14 +162,14 @@ HISTORY of the LZMA SDK
9.35 beta 2014-12-07
--------------------------
+-------------------------
- 7zr.exe now support AES encryption.
- SFX modules were added to LZMA SDK
- Some bugs were fixed.
9.21 beta 2011-04-11
--------------------------
+-------------------------
- New class FString for file names at file systems.
- Speed optimization in CRC code for big-endian CPUs.
- The BUG in Lzma2Dec.c was fixed:
@@ -155,7 +177,7 @@ HISTORY of the LZMA SDK
9.18 beta 2010-11-02
--------------------------
+-------------------------
- New small SFX module for installers (SfxSetup).
@@ -168,7 +190,7 @@ HISTORY of the LZMA SDK
9.11 beta 2010-03-15
-------------------------
- PPMd compression method support
-
+
9.09 2009-12-12
-------------------------
@@ -202,7 +224,7 @@ HISTORY of the LZMA SDK
4.61 beta 2008-11-23
-------------------------
- The bug in ANSI-C LZMA Decoder was fixed:
- If encoded stream was corrupted, decoder could access memory
+ If encoded stream was corrupted, decoder could access memory
outside of allocated range.
- Some changes in ANSI-C 7z Decoder interfaces.
- LZMA SDK is placed in the public domain.
@@ -216,7 +238,7 @@ HISTORY of the LZMA SDK
4.59 beta 2008-08-13
-------------------------
- The bug was fixed:
- LZMA Encoder in fast compression mode could access memory outside of
+ LZMA Encoder in fast compression mode could access memory outside of
allocated range in some rare cases.
@@ -229,7 +251,7 @@ HISTORY of the LZMA SDK
4.57 2007-12-12
-------------------------
-- Speed optimizations in ?++ LZMA Decoder.
+- Speed optimizations in ?++ LZMA Decoder.
- Small changes for more compatibility with some C/C++ compilers.
@@ -239,36 +261,36 @@ HISTORY of the LZMA SDK
- now it supports BCJ and BCJ2 filters
- now it supports files larger than 4 GB.
- now it supports "Last Write Time" field for files.
-- C++ code for .7z archives compressing/decompressing from 7-zip
+- C++ code for .7z archives compressing/decompressing from 7-zip
was included to LZMA SDK.
-
+
4.43 2006-06-04
-------------------------
- Small changes for more compatibility with some C/C++ compilers.
-
+
4.42 2006-05-15
-------------------------
- Small changes in .h files in ANSI-C version.
-
+
4.39 beta 2006-04-14
-------------------------
- The bug in versions 4.33b:4.38b was fixed:
- C++ version of LZMA encoder could not correctly compress
+ C++ version of LZMA encoder could not correctly compress
files larger than 2 GB with HC4 match finder (-mfhc4).
-
+
4.37 beta 2005-04-06
-------------------------
-- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was defined.
+- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was defined.
4.35 beta 2005-03-02
-------------------------
- The bug was fixed in C++ version of LZMA Decoder:
- If encoded stream was corrupted, decoder could access memory
+ If encoded stream was corrupted, decoder could access memory
outside of allocated range.
@@ -339,7 +361,7 @@ HISTORY of the LZMA SDK
4.17 2005-04-18
-------------------------
-- New example for RAM->RAM compressing/decompressing:
+- New example for RAM->RAM compressing/decompressing:
LZMA + BCJ (filter for x86 code):
- LzmaRam.h
- LzmaRam.cpp
@@ -350,11 +372,11 @@ HISTORY of the LZMA SDK
4.16 2005-03-29
-------------------------
-- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):
+- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):
If _LZMA_OUT_READ was defined, and if encoded stream was corrupted,
decoder could access memory outside of allocated range.
- Speed optimization of ANSI-C LZMA Decoder (now it's about 20% faster).
- Old version of LZMA Decoder now is in file LzmaDecodeSize.c.
+ Old version of LZMA Decoder now is in file LzmaDecodeSize.c.
LzmaDecodeSize.c can provide slightly smaller code than LzmaDecode.c
- Small speed optimization in LZMA C++ code
- filter for SPARC's code was added
@@ -369,7 +391,7 @@ HISTORY of the LZMA SDK
4.05 2004-08-25
-------------------------
-- Source code of filters for x86, IA-64, ARM, ARM-Thumb
+- Source code of filters for x86, IA-64, ARM, ARM-Thumb
and PowerPC code was included to SDK
- Some internal minor changes
@@ -381,8 +403,8 @@ HISTORY of the LZMA SDK
4.03 2004-06-18
-------------------------
-- "Benchmark" command was added. It measures compressing
- and decompressing speed and shows rating values.
+- "Benchmark" command was added. It measures compressing
+ and decompressing speed and shows rating values.
Also it checks hardware errors.
@@ -411,7 +433,7 @@ HISTORY of the LZMA SDK
HISTORY of the LZMA
-------------------
- 2001-2008: Improvements to LZMA compressing/decompressing code,
+ 2001-2008: Improvements to LZMA compressing/decompressing code,
keeping compatibility with original LZMA format
1996-2001: Development of LZMA compression format
@@ -419,6 +441,6 @@ HISTORY of the LZMA
2001-08-30: LZMA compression was added to 7-Zip
1999-01-02: First version of 7-Zip was released
-
+
End of document
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
index a3deb20943..e17cd4fe1a 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
@@ -1,4 +1,4 @@
-LZMA SDK 18.05
+LZMA SDK 19.00
--------------
LZMA SDK provides the documentation, samples, header files,
--
2.29.2.windows.3
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
@ 2020-12-15 2:06 Wei Liu
2020-12-17 1:30 ` 回复: " gaoliming
0 siblings, 1 reply; 7+ messages in thread
From: Wei Liu @ 2020-12-15 2:06 UTC (permalink / raw)
To: devel; +Cc: Wei Liu, Liming Gao, Laszlo Ersek
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
New formal release in https://www.7-zip.org/sdk.html is 19.00.
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Wei Liu <weix.c.liu@intel.com>
---
BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4 ++--
BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8 ++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5 +++--
BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8 ++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9 +++++----
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 164 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14 +++++++-------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32 ++++++++++++++++----------------
BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 733 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 22 ++++++++++++++++++++++
BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
12 files changed, 710 insertions(+), 408 deletions(-)
diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
index 7cf967a774..b2338e2598 100644
--- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
+++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
@@ -1,3 +1,3 @@
-LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
-was placed in the public domain on 2018-04-30. It was
+LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
+was placed in the public domain on 2019-02-21. It was
released on the http://www.7-zip.org/sdk.html website.
diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
index bebdb9aa84..bf4706ee23 100644
--- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
+++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
@@ -1,9 +1,9 @@
/** @file
LZMA Compress/Decompress tool (LzmaCompress)
- Based on LZMA SDK 18.05:
+ Based on LZMA SDK 19.00:
LzmaUtil.c -- Test application for LZMA compression
- 2018-04-30 : Igor Pavlov : Public domain
+ 2019-02-21 : Igor Pavlov : Public domain
Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
SPDX-License-Identifier: BSD-2-Clause-Patent
@@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate memory";
const char *kDataErrorMessage = "Data error";
const char *kInvalidParamValMessage = "Invalid parameter value";
-static Bool mQuietMode = False;
+static BoolInt mQuietMode = False;
static CONVERTER_TYPE mConType = NoConverter;
UINT64 mDictionarySize = 28;
@@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
CFileOutStream outStream;
int res;
int encodeMode = 0;
- Bool modeWasSet = False;
+ BoolInt modeWasSet = False;
const char *inputFile = NULL;
const char *outputFile = "file.tmp";
int param;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
index 4977cdaa66..593f5aa259 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
@@ -1,5 +1,5 @@
/* 7zTypes.h -- Basic types
-2017-07-17 : Igor Pavlov : Public domain */
+2018-08-04 : Igor Pavlov : Public domain */
#ifndef __7Z_TYPES_H
#define __7Z_TYPES_H
@@ -103,7 +103,8 @@ typedef UInt32 SizeT;
typedef size_t SizeT;
#endif
-typedef int Bool;
+typedef int BoolInt;
+/* typedef BoolInt Bool; */
#define True 1
#define False 0
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
index ed3aa94270..0074c64be9 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
@@ -1,7 +1,7 @@
-#define MY_VER_MAJOR 18
-#define MY_VER_MINOR 05
+#define MY_VER_MAJOR 19
+#define MY_VER_MINOR 00
#define MY_VER_BUILD 0
-#define MY_VERSION_NUMBERS "18.05"
+#define MY_VERSION_NUMBERS "19.00"
#define MY_VERSION MY_VERSION_NUMBERS
#ifdef MY_CPU_NAME
@@ -10,7 +10,7 @@
#define MY_VERSION_CPU MY_VERSION
#endif
-#define MY_DATE "2018-04-30"
+#define MY_DATE "2019-02-21"
#undef MY_COPYRIGHT
#undef MY_VERSION_COPYRIGHT_DATE
#define MY_AUTHOR_NAME "Igor Pavlov"
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
index 7fb27282c7..5f74c1c0cb 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
@@ -1,5 +1,5 @@
/* CpuArch.h -- CPU specific code
-2017-09-04 : Igor Pavlov : Public domain */
+2018-02-18 : Igor Pavlov : Public domain */
#ifndef __CPU_ARCH_H
#define __CPU_ARCH_H
@@ -318,15 +318,16 @@ enum
void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d);
-Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
+BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
int x86cpuid_GetFirm(const Cx86cpuid *p);
#define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) & 0xF))
#define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) & 0xF))
#define x86cpuid_GetStepping(ver) (ver & 0xF)
-Bool CPU_Is_InOrder();
-Bool CPU_Is_Aes_Supported();
+BoolInt CPU_Is_InOrder();
+BoolInt CPU_Is_Aes_Supported();
+BoolInt CPU_IsSupported_PageGB();
#endif
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
index 6ea82a9b53..4eefc17dd2 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
@@ -1,5 +1,5 @@
/* LzFind.c -- Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-07-08 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -138,7 +138,7 @@ static void MatchFinder_SetDefaultSettings(CMatchFinder *p)
void MatchFinder_Construct(CMatchFinder *p)
{
- UInt32 i;
+ unsigned i;
p->bufferBase = NULL;
p->directInput = 0;
p->hash = NULL;
@@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
for (i = 0; i < 256; i++)
{
- UInt32 r = i;
+ UInt32 r = (UInt32)i;
unsigned j;
for (j = 0; j < 8; j++)
r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
@@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
MatchFinder_ReduceOffsets(p, subValue);
}
+
+MY_NO_INLINE
static void MatchFinder_CheckLimits(CMatchFinder *p)
{
if (p->pos == kMaxValForNormalize)
@@ -379,10 +381,16 @@ static void MatchFinder_CheckLimits(CMatchFinder *p)
MatchFinder_SetLimits(p);
}
-static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
+
+/*
+ (lenLimit > maxLen)
+*/
+MY_FORCE_INLINE
+static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
- UInt32 *distances, UInt32 maxLen)
+ UInt32 *distances, unsigned maxLen)
{
+ /*
son[_cyclicBufferPos] = curMatch;
for (;;)
{
@@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = len;
*distances++ = delta - 1;
if (len == lenLimit)
return distances;
@@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
}
}
}
+ */
+
+ const Byte *lim = cur + lenLimit;
+ son[_cyclicBufferPos] = curMatch;
+ do
+ {
+ UInt32 delta = pos - curMatch;
+ if (delta >= _cyclicBufferSize)
+ break;
+ {
+ ptrdiff_t diff;
+ curMatch = son[_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)];
+ diff = (ptrdiff_t)0 - delta;
+ if (cur[maxLen] == cur[maxLen + diff])
+ {
+ const Byte *c = cur;
+ while (*c == c[diff])
+ {
+ if (++c == lim)
+ {
+ distances[0] = (UInt32)(lim - cur);
+ distances[1] = delta - 1;
+ return distances + 2;
+ }
+ }
+ {
+ unsigned len = (unsigned)(c - cur);
+ if (maxLen < len)
+ {
+ maxLen = len;
+ distances[0] = (UInt32)len;
+ distances[1] = delta - 1;
+ distances += 2;
+ }
+ }
+ }
+ }
+ }
+ while (--cutValue);
+
+ return distances;
}
+
+MY_FORCE_INLINE
UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
UInt32 *distances, UInt32 maxLen)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
return distances;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = pair[0];
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = (UInt32)len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
+ *ptr1 = pair0;
*ptr0 = pair[1];
return distances;
}
@@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byt
static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
{
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
for (;;)
{
UInt32 delta = pos - curMatch;
@@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
return;
}
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
if (pb[len] == cur[len])
{
while (++len != lenLimit)
@@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const
p->buffer++; \
if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
-#define MOVE_POS_RET MOVE_POS return offset;
+#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define GET_MATCHES_HEADER2(minLen, ret_op) \
- UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
- lenLimit = p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
+ unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
+ lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
cur = p->buffer;
#define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen, return 0)
@@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
#define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos, p->cyclicBufferSize, p->cutValue
#define GET_MATCHES_FOOTER(offset, maxLen) \
- offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
- distances + offset, maxLen) - distances); MOVE_POS_RET;
+ offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch, MF_PARAMS(p), \
+ distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
#define SKIP_FOOTER \
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
#define UPDATE_maxLen { \
ptrdiff_t diff = (ptrdiff_t)0 - d2; \
const Byte *c = cur + maxLen; \
const Byte *lim = cur + lenLimit; \
for (; c != lim; c++) if (*(c + diff) != *c) break; \
- maxLen = (UInt32)(c - cur); }
+ maxLen = (unsigned)(c - cur); }
static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(2)
HASH2_CALC;
curMatch = p->hash[hv];
@@ -561,7 +615,7 @@ static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
@@ -572,7 +626,8 @@ UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, d2, maxLen, offset, pos;
+ UInt32 h2, d2, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(3)
@@ -594,12 +649,12 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
UPDATE_maxLen
- distances[0] = maxLen;
+ distances[0] = (UInt32)maxLen;
distances[1] = d2 - 1;
offset = 2;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
@@ -609,7 +664,8 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -618,12 +674,12 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[hv] = pos;
@@ -632,7 +688,8 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
@@ -648,10 +705,10 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
- SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
+ SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
MOVE_POS_RET;
}
}
@@ -674,13 +731,13 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
d4 = pos - (hash + kFix4HashSize)[h4];
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[h4] = pos;
(hash + kFix5HashSize)[hv] = pos;
@@ -741,7 +798,8 @@ static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 h2, h3, d2, d3, maxLen, offset, pos;
+ UInt32 h2, h3, d2, d3, pos;
+ unsigned maxLen, offset;
UInt32 *hash;
GET_MATCHES_HEADER(4)
@@ -750,12 +808,11 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
-
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[hv] = pos;
@@ -764,7 +821,8 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
{
- distances[0] = maxLen = 2;
+ maxLen = 2;
+ distances[0] = 2;
distances[1] = d2 - 1;
offset = 2;
}
@@ -780,7 +838,7 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (offset != 0)
{
UPDATE_maxLen
- distances[(size_t)offset - 2] = maxLen;
+ distances[(size_t)offset - 2] = (UInt32)maxLen;
if (maxLen == lenLimit)
{
p->son[p->cyclicBufferPos] = curMatch;
@@ -791,7 +849,7 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
if (maxLen < 3)
maxLen = 3;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances + offset, maxLen) - (distances));
MOVE_POS_RET
}
@@ -808,13 +866,13 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
hash = p->hash;
pos = p->pos;
- d2 = pos - hash[ h2];
+ d2 = pos - hash [h2];
d3 = pos - (hash + kFix3HashSize)[h3];
d4 = pos - (hash + kFix4HashSize)[h4];
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] = pos;
+ hash [h2] = pos;
(hash + kFix3HashSize)[h3] = pos;
(hash + kFix4HashSize)[h4] = pos;
(hash + kFix5HashSize)[hv] = pos;
@@ -877,12 +935,12 @@ static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
{
- UInt32 offset;
+ unsigned offset;
GET_MATCHES_HEADER(3)
HASH_ZIP_CALC;
curMatch = p->hash[hv];
p->hash[hv] = p->pos;
- offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
+ offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
distances, 2) - (distances));
MOVE_POS_RET
}
@@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH4_CALC;
hash = p->hash;
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[hv] = p->pos;
SKIP_FOOTER
@@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH5_CALC;
hash = p->hash;
curMatch = (hash + kFix5HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[h4] =
(hash + kFix5HashSize)[hv] = p->pos;
@@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH4_CALC;
hash = p->hash;
curMatch = (hash + kFix4HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[hv] = p->pos;
p->son[p->cyclicBufferPos] = curMatch;
@@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
HASH5_CALC;
hash = p->hash;
curMatch = hash + kFix5HashSize)[hv];
- hash[ h2] =
+ hash [h2] =
(hash + kFix3HashSize)[h3] =
(hash + kFix4HashSize)[h4] =
(hash + kFix5HashSize)[hv] = p->pos;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
index 2563824fcd..df32146f92 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
@@ -1,5 +1,5 @@
/* LzFindMt.c -- multithreaded Match finder for LZ algorithms
-2017-06-10 : Igor Pavlov : Public domain */
+2018-12-29 : Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -232,38 +232,57 @@ static void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
#define kEmptyHashValue 0
-/* #define MFMT_GM_INLINE */
+#define MFMT_GM_INLINE
#ifdef MFMT_GM_INLINE
-#define NO_INLINE MY_FAST_CALL
+/*
+ we use size_t for _cyclicBufferPos instead of UInt32
+ to eliminate "movsx" BUG in old MSVC x64 compiler.
+*/
-static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
- UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
- UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit, UInt32 size, UInt32 *posRes)
+MY_NO_INLINE
+static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *cur, CLzRef *son,
+ size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
+ UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32 *limit, UInt32 size, UInt32 *posRes)
{
do
{
- UInt32 *distances = _distances + 1;
- UInt32 curMatch = pos - *hash++;
+ UInt32 *_distances = ++distances;
+ UInt32 delta = *hash++;
- CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
- CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
- UInt32 len0 = 0, len1 = 0;
+ CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
+ CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
+ unsigned len0 = 0, len1 = 0;
UInt32 cutValue = _cutValue;
- UInt32 maxLen = _maxLen;
- for (;;)
+ unsigned maxLen = (unsigned)_maxLen;
+
+ /*
+ if (size > 1)
{
- UInt32 delta = pos - curMatch;
- if (cutValue-- == 0 || delta >= _cyclicBufferSize)
+ UInt32 delta = *hash;
+ if (delta < _cyclicBufferSize)
{
- *ptr0 = *ptr1 = kEmptyHashValue;
- break;
+ UInt32 cyc1 = _cyclicBufferPos + 1;
+ CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ? _cyclicBufferSize : 0)) << 1);
+ Byte b = *(cur + 1 - delta);
+ _distances[0] = pair[0];
+ _distances[1] = b;
}
+ }
+ */
+ if (cutValue == 0 || delta >= _cyclicBufferSize)
+ {
+ *ptr0 = *ptr1 = kEmptyHashValue;
+ }
+ else
+ for(;;)
+ {
{
- CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
+ CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
const Byte *pb = cur - delta;
- UInt32 len = (len0 < len1 ? len0 : len1);
+ unsigned len = (len0 < len1 ? len0 : len1);
+ UInt32 pair0 = *pair;
if (pb[len] == cur[len])
{
if (++len != lenLimit && pb[len] == cur[len])
@@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *
break;
if (maxLen < len)
{
- *distances++ = maxLen = len;
+ maxLen = len;
+ *distances++ = (UInt32)len;
*distances++ = delta - 1;
if (len == lenLimit)
{
- *ptr1 = pair[0];
- *ptr0 = pair[1];
+ UInt32 pair1 = pair[1];
+ *ptr1 = pair0;
+ *ptr0 = pair1;
break;
}
}
}
- if (pb[len] < cur[len])
- {
- *ptr1 = curMatch;
- ptr1 = pair + 1;
- curMatch = *ptr1;
- len1 = len;
- }
- else
{
- *ptr0 = curMatch;
- ptr0 = pair;
- curMatch = *ptr0;
- len0 = len;
+ UInt32 curMatch = pos - delta;
+ // delta = pos - *pair;
+ // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >> 31];
+ if (pb[len] < cur[len])
+ {
+ delta = pos - pair[1];
+ *ptr1 = curMatch;
+ ptr1 = pair + 1;
+ len1 = len;
+ }
+ else
+ {
+ delta = pos - *pair;
+ *ptr0 = curMatch;
+ ptr0 = pair;
+ len0 = len;
+ }
}
}
+ if (--cutValue == 0 || delta >= _cyclicBufferSize)
+ {
+ *ptr0 = *ptr1 = kEmptyHashValue;
+ break;
+ }
}
pos++;
_cyclicBufferPos++;
cur++;
{
UInt32 num = (UInt32)(distances - _distances);
- *_distances = num - 1;
- _distances += num;
- limit -= num;
+ _distances[-1] = num;
}
}
- while (limit > 0 && --size != 0);
+ while (distances < limit && --size != 0);
*posRes = pos;
- return limit;
+ return distances;
}
#endif
+
+
static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
{
UInt32 numProcessed = 0;
UInt32 curPos = 2;
- UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
+ UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
distances[1] = p->hashNumAvail;
@@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
#else
{
UInt32 posRes;
- curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
- distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
+ curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
+ distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBufPos,
+ distances + limit,
+ size, &posRes) - distances);
p->hashBufPos += posRes - pos;
cyclicBufferPos += posRes - pos;
p->buffer += posRes - pos;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
index 3d86c788f3..fdd17008c2 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
@@ -1,5 +1,5 @@
/* LzFindMt.h -- multithreaded Match finder for LZ algorithms
-2017-04-03 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#ifndef __LZ_FIND_MT_H
#define __LZ_FIND_MT_H
@@ -19,10 +19,10 @@ EXTERN_C_BEGIN
typedef struct _CMtSync
{
- Bool wasCreated;
- Bool needStart;
- Bool exit;
- Bool stopWriting;
+ BoolInt wasCreated;
+ BoolInt needStart;
+ BoolInt exit;
+ BoolInt stopWriting;
CThread thread;
CAutoResetEvent canStart;
@@ -30,8 +30,8 @@ typedef struct _CMtSync
CAutoResetEvent wasStopped;
CSemaphore freeSemaphore;
CSemaphore filledSemaphore;
- Bool csWasInitialized;
- Bool csWasEntered;
+ BoolInt csWasInitialized;
+ BoolInt csWasEntered;
CCriticalSection cs;
UInt32 numProcessedBlocks;
} CMtSync;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
index 962b94bb63..4d1576419f 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
@@ -1,13 +1,13 @@
/* LzmaDec.c -- LZMA Decoder
-2018-02-28 : Igor Pavlov : Public domain */
+2018-07-04 : Igor Pavlov : Public domain */
#include "Precomp.h"
+#include <string.h>
+
/* #include "CpuArch.h" */
#include "LzmaDec.h"
-#include <string.h>
-
#define kNumTopBits 24
#define kTopValue ((UInt32)1 << kNumTopBits)
@@ -19,7 +19,7 @@
#define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
#define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt - (ttt >> kNumMoveBits));
#define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
@@ -66,7 +66,7 @@
#define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit) return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
-#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
+#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
#define UPDATE_0_CHECK range = bound;
#define UPDATE_1_CHECK range -= bound; code -= bound;
#define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
@@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
curLen = ((rem < len) ? (unsigned)rem : len);
pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
- processedPos += curLen;
+ processedPos += (UInt32)curLen;
len -= curLen;
if (curLen <= dicBufSize - pos)
@@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
Byte *dest = dic + dicPos;
ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
const Byte *lim = dest + curLen;
- dicPos += curLen;
+ dicPos += (SizeT)curLen;
do
*(dest) = (Byte)*(dest + src);
while (++dest != lim);
@@ -572,14 +572,14 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
p->buf = buf;
p->range = range;
p->code = code;
- p->remainLen = len;
+ p->remainLen = (UInt32)len;
p->dicPos = dicPos;
p->processedPos = processedPos;
p->reps[0] = rep0;
p->reps[1] = rep1;
p->reps[2] = rep2;
p->reps[3] = rep3;
- p->state = state;
+ p->state = (UInt32)state;
return SZ_OK;
}
@@ -601,8 +601,8 @@ static void MY_FAST_CALL LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
p->checkDicSize = p->prop.dicSize;
- p->processedPos += len;
- p->remainLen -= len;
+ p->processedPos += (UInt32)len;
+ p->remainLen -= (UInt32)len;
while (len != 0)
{
len--;
@@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, const Byte *buf, SizeT inS
}
-void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
+void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt initState)
{
p->remainLen = kMatchSpecLenStart + 1;
p->tempBufSize = 0;
@@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
p->tempBufSize = rem;
if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
{
- int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
+ int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, (SizeT)rem);
if (dummyRes == DUMMY_ERROR)
{
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
*status = LZMA_STATUS_NEEDS_MORE_INPUT;
return SZ_OK;
}
@@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *sr
return SZ_ERROR_FAIL; /* some internal error */
lookAhead -= rem;
}
- (*srcLen) += lookAhead;
+ (*srcLen) += (SizeT)lookAhead;
src += lookAhead;
- inSize -= lookAhead;
+ inSize -= (SizeT)lookAhead;
p->tempBufSize = 0;
}
}
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
index e281716fee..14086fc4f9 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
@@ -1,5 +1,5 @@
/* LzmaEnc.c -- LZMA Encoder
-2018-04-29 : Igor Pavlov : Public domain */
+2019-01-10: Igor Pavlov : Public domain */
#include "Precomp.h"
@@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
#define kNumBitPriceShiftBits 4
#define kBitPrice (1 << kNumBitPriceShiftBits)
+#define REP_LEN_COUNT 64
+
void LzmaEncProps_Init(CLzmaEncProps *p)
{
p->level = 5;
@@ -183,8 +185,11 @@ typedef struct
} COptimal;
-#define kNumOpts (1 << 12)
-#define kPackReserve (1 + kNumOpts * 2)
+// 18.06
+#define kNumOpts (1 << 11)
+#define kPackReserve (kNumOpts * 8)
+// #define kNumOpts (1 << 12)
+// #define kPackReserve (1 + kNumOpts * 2)
#define kNumLenToPosStates 4
#define kNumPosSlotBits 6
@@ -236,10 +241,18 @@ typedef struct
typedef struct
{
unsigned tableSize;
- unsigned counters[LZMA_NUM_PB_STATES_MAX];
UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
+ // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols * 2];
+ // UInt32 prices2[kLenNumSymbolsTotal];
} CLenPriceEnc;
+#define GET_PRICE_LEN(p, posState, len) \
+ ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
+
+/*
+#define GET_PRICE_LEN(p, posState, len) \
+ ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) - 2) & (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >> 9)))
+*/
typedef struct
{
@@ -308,16 +321,18 @@ typedef struct
unsigned lc, lp, pb;
unsigned lclp;
- Bool fastMode;
- Bool writeEndMark;
- Bool finished;
- Bool multiThread;
- Bool needInit;
+ BoolInt fastMode;
+ BoolInt writeEndMark;
+ BoolInt finished;
+ BoolInt multiThread;
+ BoolInt needInit;
+ // BoolInt _maxMode;
UInt64 nowPos64;
unsigned matchPriceCount;
- unsigned alignPriceCount;
+ // unsigned alignPriceCount;
+ int repLenEncCounter;
unsigned distTableSize;
@@ -325,7 +340,7 @@ typedef struct
SRes result;
#ifndef _7ZIP_ST
- Bool mtMode;
+ BoolInt mtMode;
// begin of CMatchFinderMt is used in LZ thread
CMatchFinderMt matchFinderMt;
// end of CMatchFinderMt is used in BT and HASH threads
@@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEncProps *props2)
p->lp = props.lp;
p->pb = props.pb;
p->fastMode = (props.algo == 0);
+ // p->_maxMode = True;
p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
{
unsigned numHashBytes = 4;
@@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
#ifdef _LZMA_ENC_USE_BRANCH
-#define RC_BIT(p, prob, symbol) { \
+#define RC_BIT(p, prob, bit) { \
RC_BIT_PRE(p, prob) \
- if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >> kNumMoveBits; } \
+ if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >> kNumMoveBits; } \
else { (p)->low += newBound; range -= newBound; ttt -= ttt >> kNumMoveBits; } \
*(prob) = (CLzmaProb)ttt; \
RC_NORM(p) \
@@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
#else
-#define RC_BIT(p, prob, symbol) { \
+#define RC_BIT(p, prob, bit) { \
UInt32 mask; \
RC_BIT_PRE(p, prob) \
- mask = 0 - (UInt32)symbol; \
+ mask = 0 - (UInt32)bit; \
range &= mask; \
mask &= newBound; \
range -= mask; \
(p)->low += mask; \
- mask = (UInt32)symbol - 1; \
+ mask = (UInt32)bit - 1; \
range += newBound & mask; \
mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
mask += ((1 << kNumMoveBits) - 1); \
@@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p, CLzmaProb *prob)
p->range = range;
}
-static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
+static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
{
UInt32 range = p->range;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
UInt32 ttt, newBound;
- // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
- CLzmaProb *prob = probs + (symbol >> 8);
- UInt32 bit = (symbol >> 7) & 1;
- symbol <<= 1;
+ // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
+ CLzmaProb *prob = probs + (sym >> 8);
+ UInt32 bit = (sym >> 7) & 1;
+ sym <<= 1;
RC_BIT(p, prob, bit);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
p->range = range;
}
-static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol, UInt32 matchByte)
+static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 sym, UInt32 matchByte)
{
UInt32 range = p->range;
UInt32 offs = 0x100;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
UInt32 ttt, newBound;
CLzmaProb *prob;
UInt32 bit;
matchByte <<= 1;
- // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol >> 8)), (symbol >> 7) & 1);
- prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
- bit = (symbol >> 7) & 1;
- symbol <<= 1;
- offs &= ~(matchByte ^ symbol);
+ // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >> 8)), (sym >> 7) & 1);
+ prob = probs + (offs + (matchByte & offs) + (sym >> 8));
+ bit = (sym >> 7) & 1;
+ sym <<= 1;
+ offs &= ~(matchByte ^ sym);
RC_BIT(p, prob, bit);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
p->range = range;
}
@@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice *ProbPrices)
}
-#define GET_PRICE(prob, symbol) \
- p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
+#define GET_PRICE(prob, bit) \
+ p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
-#define GET_PRICEa(prob, symbol) \
- ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
+#define GET_PRICEa(prob, bit) \
+ ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
#define GET_PRICE_0(prob) p->ProbPrices[(prob) >> kNumMoveReducingBits]
#define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
@@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice *ProbPrices)
#define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
-static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol, const CProbPrice *ProbPrices)
+static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const CProbPrice *ProbPrices)
{
UInt32 price = 0;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
- unsigned bit = symbol & 1;
- symbol >>= 1;
- price += GET_PRICEa(probs[symbol], bit);
+ unsigned bit = sym & 1;
+ sym >>= 1;
+ price += GET_PRICEa(probs[sym], bit);
}
- while (symbol >= 2);
+ while (sym >= 2);
return price;
}
-static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32 symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
+static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32 sym, UInt32 matchByte, const CProbPrice *ProbPrices)
{
UInt32 price = 0;
UInt32 offs = 0x100;
- symbol |= 0x100;
+ sym |= 0x100;
do
{
matchByte <<= 1;
- price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)], (symbol >> 7) & 1);
- symbol <<= 1;
- offs &= ~(matchByte ^ symbol);
+ price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)], (sym >> 7) & 1);
+ sym <<= 1;
+ offs &= ~(matchByte ^ sym);
}
- while (symbol < 0x10000);
+ while (sym < 0x10000);
return price;
}
-static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, unsigned numBits, UInt32 symbol)
+static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, unsigned numBits, unsigned sym)
{
UInt32 range = rc->range;
unsigned m = 1;
do
{
UInt32 ttt, newBound;
- unsigned bit = symbol & 1;
+ unsigned bit = sym & 1;
// RangeEnc_EncodeBit(rc, probs + m, bit);
- symbol >>= 1;
+ sym >>= 1;
RC_BIT(rc, probs + m, bit);
m = (m << 1) | bit;
}
@@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
p->high[i] = kProbInitValue;
}
-static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol, unsigned posState)
+static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym, unsigned posState)
{
UInt32 range, ttt, newBound;
CLzmaProb *probs = p->low;
range = rc->range;
RC_BIT_PRE(rc, probs);
- if (symbol >= kLenNumLowSymbols)
+ if (sym >= kLenNumLowSymbols)
{
RC_BIT_1(rc, probs);
probs += kLenNumLowSymbols;
RC_BIT_PRE(rc, probs);
- if (symbol >= kLenNumLowSymbols * 2)
+ if (sym >= kLenNumLowSymbols * 2)
{
RC_BIT_1(rc, probs);
rc->range = range;
- // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol - kLenNumLowSymbols * 2);
- LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
+ // RcTree_Encode(rc, p->high, kLenNumHighBits, sym - kLenNumLowSymbols * 2);
+ LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
return;
}
- symbol -= kLenNumLowSymbols;
+ sym -= kLenNumLowSymbols;
}
- // RcTree_Encode(rc, probs + (posState << kLenNumLowBits), kLenNumLowBits, symbol);
+ // RcTree_Encode(rc, probs + (posState << kLenNumLowBits), kLenNumLowBits, sym);
{
unsigned m;
unsigned bit;
RC_BIT_0(rc, probs);
probs += (posState << (1 + kLenNumLowBits));
- bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
- bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
- bit = symbol & 1; RC_BIT(rc, probs + m, bit);
+ bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
+ bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
+ bit = sym & 1; RC_BIT(rc, probs + m, bit);
rc->range = range;
}
}
@@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs, UInt32 startPrice, UInt32 *price
}
-MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
- CLenPriceEnc *p, unsigned posState,
+MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
+ CLenPriceEnc *p,
+ unsigned numPosStates,
const CLenEnc *enc,
const CProbPrice *ProbPrices)
{
- // int y; for (y = 0; y < 100; y++) {
- UInt32 a;
- unsigned i, numSymbols;
+ UInt32 b;
+
+ {
+ unsigned prob = enc->low[0];
+ UInt32 a, c;
+ unsigned posState;
+ b = GET_PRICEa_1(prob);
+ a = GET_PRICEa_0(prob);
+ c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
+ for (posState = 0; posState < numPosStates; posState++)
+ {
+ UInt32 *prices = p->prices[posState];
+ const CLzmaProb *probs = enc->low + (posState << (1 + kLenNumLowBits));
+ SetPrices_3(probs, a, prices, ProbPrices);
+ SetPrices_3(probs + kLenNumLowSymbols, c, prices + kLenNumLowSymbols, ProbPrices);
+ }
+ }
- UInt32 *prices = p->prices[posState];
+ /*
{
- const CLzmaProb *probs = enc->low + (posState << (1 + kLenNumLowBits));
- SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
+ unsigned i;
+ UInt32 b;
+ a = GET_PRICEa_0(enc->low[0]);
+ for (i = 0; i < kLenNumLowSymbols; i++)
+ p->prices2[i] = a;
a = GET_PRICEa_1(enc->low[0]);
- SetPrices_3(probs + kLenNumLowSymbols, a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices + kLenNumLowSymbols, ProbPrices);
+ b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
+ for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
+ p->prices2[i] = b;
a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
}
- numSymbols = p->tableSize;
- p->counters[posState] = numSymbols;
- for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
+ */
+
+ // p->counter = numSymbols;
+ // p->counter = 64;
+
{
- prices[i] = a +
- // RcTree_GetPrice(enc->high, kLenNumHighBits, i - kLenNumLowSymbols * 2, ProbPrices);
- LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2, ProbPrices);
- /*
- unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
- UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1, sym, ProbPrices);
- UInt32 prob = enc->high[(1 << 7) + sym];
- prices[i ] = price + GET_PRICEa_0(prob);
- prices[i + 1] = price + GET_PRICEa_1(prob);
- */
- }
- // }
-}
+ unsigned i = p->tableSize;
+
+ if (i > kLenNumLowSymbols * 2)
+ {
+ const CLzmaProb *probs = enc->high;
+ UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
+ i -= kLenNumLowSymbols * 2 - 1;
+ i >>= 1;
+ b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
+ do
+ {
+ /*
+ p->prices2[i] = a +
+ // RcTree_GetPrice(enc->high, kLenNumHighBits, i - kLenNumLowSymbols * 2, ProbPrices);
+ LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
+ */
+ // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1, sym, ProbPrices);
+ unsigned sym = --i + (1 << (kLenNumHighBits - 1));
+ UInt32 price = b;
+ do
+ {
+ unsigned bit = sym & 1;
+ sym >>= 1;
+ price += GET_PRICEa(probs[sym], bit);
+ }
+ while (sym >= 2);
-static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
- const CLenEnc *enc,
- const CProbPrice *ProbPrices)
-{
- unsigned posState;
- for (posState = 0; posState < numPosStates; posState++)
- LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
-}
+ {
+ unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits - 1))];
+ prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
+ prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
+ }
+ }
+ while (i);
+ {
+ unsigned posState;
+ size_t num = (p->tableSize - kLenNumLowSymbols * 2) * sizeof(p->prices[0][0]);
+ for (posState = 1; posState < numPosStates; posState++)
+ memcpy(p->prices[posState] + kLenNumLowSymbols * 2, p->prices[0] + kLenNumLowSymbols * 2, num);
+ }
+ }
+ }
+}
/*
#ifdef SHOW_STAT
@@ -917,7 +976,7 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned numPosStates,
#define MOVE_POS(p, num) { \
p->additionalOffset += (num); \
- p->matchFinder.Skip(p->matchFinderObj, (num)); }
+ p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
@@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
const Byte *p2 = p1 + len;
ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
const Byte *lim = p1 + numAvail;
- for (; p2 != lim && *p2 == p2[dif]; p2++);
+ for (; p2 != lim && *p2 == p2[dif]; p2++)
+ {}
return (unsigned)(p2 - p1);
}
}
@@ -977,7 +1037,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
+ GET_PRICE_1(p->isRep[state]) \
+ GET_PRICE_0(p->isRepG0[state])
-
+MY_FORCE_INLINE
static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex, size_t state, size_t posState)
{
UInt32 price;
@@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p, unsigned cur)
for (;;)
{
UInt32 dist = p->opt[cur].dist;
- UInt32 len = p->opt[cur].len;
- UInt32 extra = p->opt[cur].extra;
+ unsigned len = (unsigned)p->opt[cur].len;
+ unsigned extra = (unsigned)p->opt[cur].extra;
cur -= len;
if (extra)
{
wr--;
- p->opt[wr].len = len;
+ p->opt[wr].len = (UInt32)len;
cur -= extra;
len = extra;
if (extra == 1)
@@ -1045,7 +1105,7 @@ static unsigned Backward(CLzmaEnc *p, unsigned cur)
wr--;
p->opt[wr].dist = dist;
- p->opt[wr].len = len;
+ p->opt[wr].len = (UInt32)len;
}
}
@@ -1102,7 +1162,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
repLens[i] = 0;
continue;
}
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
repLens[i] = len;
if (len > repLens[repMaxIndex])
repMaxIndex = i;
@@ -1111,7 +1172,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (repLens[repMaxIndex] >= p->numFastBytes)
{
unsigned len;
- p->backRes = repMaxIndex;
+ p->backRes = (UInt32)repMaxIndex;
len = repLens[repMaxIndex];
MOVE_POS(p, len - 1)
return len;
@@ -1128,8 +1189,12 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
curByte = *data;
matchByte = *(data - reps[0]);
+
+ last = repLens[repMaxIndex];
+ if (last <= mainLen)
+ last = mainLen;
- if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] < 2)
+ if (last < 2 && curByte != matchByte)
{
p->backRes = MARK_LIT;
return 1;
@@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
LitEnc_Matched_GetPrice(probs, curByte, matchByte, p->ProbPrices) :
LitEnc_GetPrice(probs, curByte, p->ProbPrices));
}
-
+
MakeAs_Lit(&p->opt[1]);
matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
- if (matchByte == curByte)
+ // 18.06
+ if (matchByte == curByte && repLens[0] == 0)
{
UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, p->state, posState);
if (shortRepPrice < p->opt[1].price)
@@ -1160,16 +1226,13 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
p->opt[1].price = shortRepPrice;
MakeAs_ShortRep(&p->opt[1]);
}
+ if (last < 2)
+ {
+ p->backRes = p->opt[1].dist;
+ return 1;
+ }
}
-
- last = (mainLen >= repLens[repMaxIndex] ? mainLen : repLens[repMaxIndex]);
-
- if (last < 2)
- {
- p->backRes = p->opt[1].dist;
- return 1;
- }
-
+
p->opt[1].len = 1;
p->opt[0].reps[0] = reps[0];
@@ -1177,13 +1240,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
p->opt[0].reps[2] = reps[2];
p->opt[0].reps[3] = reps[3];
- {
- unsigned len = last;
- do
- p->opt[len--].price = kInfinityPrice;
- while (len >= 2);
- }
-
// ---------- REP ----------
for (i = 0; i < LZMA_NUM_REPS; i++)
@@ -1195,13 +1251,13 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
do
{
- UInt32 price2 = price + p->repLenEnc.prices[posState][(size_t)repLen - 2];
+ UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState, repLen);
COptimal *opt = &p->opt[repLen];
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = repLen;
- opt->dist = i;
+ opt->len = (UInt32)repLen;
+ opt->dist = (UInt32)i;
opt->extra = 0;
}
}
@@ -1211,38 +1267,41 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
// ---------- MATCH ----------
{
- unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
+ unsigned len = repLens[0] + 1;
if (len <= mainLen)
{
unsigned offs = 0;
UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[p->state]);
- while (len > matches[offs])
- offs += 2;
+ if (len < 2)
+ len = 2;
+ else
+ while (len > matches[offs])
+ offs += 2;
for (; ; len++)
{
COptimal *opt;
UInt32 dist = matches[(size_t)offs + 1];
- UInt32 price2 = normalMatchPrice + p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
+ UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc, posState, len);
unsigned lenToPosState = GetLenToPosState(len);
if (dist < kNumFullDistances)
- price2 += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
+ price += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
else
{
unsigned slot;
GetPosSlot2(dist, slot);
- price2 += p->alignPrices[dist & kAlignMask];
- price2 += p->posSlotPrices[lenToPosState][slot];
+ price += p->alignPrices[dist & kAlignMask];
+ price += p->posSlotPrices[lenToPosState][slot];
}
opt = &p->opt[len];
- if (price2 < opt->price)
+ if (price < opt->price)
{
- opt->price = price2;
- opt->len = len;
+ opt->price = price;
+ opt->len = (UInt32)len;
opt->dist = dist + LZMA_NUM_REPS;
opt->extra = 0;
}
@@ -1277,16 +1336,43 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
for (;;)
{
- UInt32 numAvail, numAvailFull;
+ unsigned numAvail;
+ UInt32 numAvailFull;
unsigned newLen, numPairs, prev, state, posState, startLen;
- UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
- Bool nextIsLit;
+ UInt32 litPrice, matchPrice, repMatchPrice;
+ BoolInt nextIsLit;
Byte curByte, matchByte;
const Byte *data;
COptimal *curOpt, *nextOpt;
if (++cur == last)
- return Backward(p, cur);
+ break;
+
+ // 18.06
+ if (cur >= kNumOpts - 64)
+ {
+ unsigned j, best;
+ UInt32 price = p->opt[cur].price;
+ best = cur;
+ for (j = cur + 1; j <= last; j++)
+ {
+ UInt32 price2 = p->opt[j].price;
+ if (price >= price2)
+ {
+ price = price2;
+ best = j;
+ }
+ }
+ {
+ unsigned delta = best - cur;
+ if (delta != 0)
+ {
+ MOVE_POS(p, delta);
+ }
+ }
+ cur = best;
+ break;
+ }
newLen = ReadMatchDistances(p, &numPairs);
@@ -1294,15 +1380,24 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
{
p->numPairs = numPairs;
p->longestMatchLen = newLen;
- return Backward(p, cur);
+ break;
}
curOpt = &p->opt[cur];
+
+ position++;
+
+ // we need that check here, if skip_items in p->opt are possible
+ /*
+ if (curOpt->price >= kInfinityPrice)
+ continue;
+ */
+
prev = cur - curOpt->len;
-
+
if (curOpt->len == 1)
{
- state = p->opt[prev].state;
+ state = (unsigned)p->opt[prev].state;
if (IsShortRep(curOpt))
state = kShortRepNextStates[state];
else
@@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (curOpt->extra)
{
- prev -= curOpt->extra;
+ prev -= (unsigned)curOpt->extra;
state = kState_RepAfterLit;
if (curOpt->extra == 1)
- state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit : kState_MatchAfterLit;
+ state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit : kState_MatchAfterLit);
}
else
{
- state = p->opt[prev].state;
+ state = (unsigned)p->opt[prev].state;
if (dist < LZMA_NUM_REPS)
state = kRepNextStates[state];
else
@@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
curByte = *data;
matchByte = *(data - reps[0]);
- position++;
posState = (position & p->pbMask);
/*
@@ -1391,13 +1485,25 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
< MATCH [ : LIT : REP_0 ]
*/
- curPrice = curOpt->price;
- litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
+ {
+ UInt32 curPrice = curOpt->price;
+ unsigned prob = p->isMatch[state][posState];
+ matchPrice = curPrice + GET_PRICE_1(prob);
+ litPrice = curPrice + GET_PRICE_0(prob);
+ }
nextOpt = &p->opt[(size_t)cur + 1];
nextIsLit = False;
- // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
+ // here we can allow skip_items in p->opt, if we don't check (nextOpt->price < kInfinityPrice)
+ // 18.new.06
+ if ((nextOpt->price < kInfinityPrice
+ // && !IsLitState(state)
+ && matchByte == curByte)
+ || litPrice > nextOpt->price
+ )
+ litPrice = 0;
+ else
{
const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
litPrice += (!IsLitState(state) ?
@@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
- matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
+ numAvailFull = p->numAvail;
+ {
+ unsigned temp = kNumOpts - 1 - cur;
+ if (numAvailFull > temp)
+ numAvailFull = (UInt32)temp;
+ }
+
+ // 18.06
// ---------- SHORT_REP ----------
- // if (IsLitState(state)) // 18.new
+ if (IsLitState(state)) // 18.new
if (matchByte == curByte)
- // if (repMatchPrice < nextOpt->price) // 18.new
- if (nextOpt->len < 2
+ if (repMatchPrice < nextOpt->price) // 18.new
+ // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
+ if (
+ // nextOpt->price >= kInfinityPrice ||
+ nextOpt->len < 2 // we can check nextOpt->len, if skip items are not allowed in p->opt
|| (nextOpt->dist != 0
- && nextOpt->extra <= 1 // 17.old
- ))
+ // && nextOpt->extra <= 1 // 17.old
+ )
+ )
{
UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state, posState);
- if (shortRepPrice <= nextOpt->price) // 17.old
- // if (shortRepPrice < nextOpt->price) // 18.new
+ // if (shortRepPrice <= nextOpt->price) // 17.old
+ if (shortRepPrice < nextOpt->price) // 18.new
{
nextOpt->price = shortRepPrice;
nextOpt->len = 1;
@@ -1436,13 +1553,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
- numAvailFull = p->numAvail;
- {
- UInt32 temp = kNumOpts - 1 - cur;
- if (numAvailFull > temp)
- numAvailFull = temp;
- }
-
if (numAvailFull < 2)
continue;
numAvail = (numAvailFull <= p->numFastBytes ? numAvailFull : p->numFastBytes);
@@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
// ---------- LIT : REP_0 ----------
- if (
- // litPrice != 0 && // 18.new
- !nextIsLit
+ if (!nextIsLit
+ && litPrice != 0 // 18.new
&& matchByte != curByte
&& numAvailFull > 2)
{
@@ -1465,8 +1574,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (limit > numAvailFull)
limit = numAvailFull;
for (len = 3; len < limit && data[len] == data2[len]; len++)
- {
- }
+ {}
{
unsigned state2 = kLiteralNextStates[state];
@@ -1474,8 +1582,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
{
unsigned offset = cur + len;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
@@ -1483,14 +1592,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
COptimal *opt;
len--;
// price2 = price + GetPrice_Len_Rep_0(p, len, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len;
+ opt->len = (UInt32)len;
opt->dist = 0;
opt->extra = 1;
}
@@ -1502,6 +1611,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
startLen = 2; /* speed optimization */
+
{
// ---------- REP ----------
unsigned repIndex = 0; // 17.old
@@ -1514,24 +1624,28 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (data[0] != data2[0] || data[1] != data2[1])
continue;
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
// if (len < startLen) continue; // 18.new: speed optimization
- while (last < cur + len)
- p->opt[++last].price = kInfinityPrice;
+ {
+ unsigned offset = cur + len;
+ if (last < offset)
+ last = offset;
+ }
{
unsigned len2 = len;
price = repMatchPrice + GetPrice_PureRep(p, repIndex, state, posState);
do
{
- UInt32 price2 = price + p->repLenEnc.prices[posState][(size_t)len2 - 2];
+ UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState, len2);
COptimal *opt = &p->opt[cur + len2];
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
- opt->dist = repIndex;
+ opt->len = (UInt32)len2;
+ opt->dist = (UInt32)repIndex;
opt->extra = 0;
}
}
@@ -1551,15 +1665,14 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (limit > numAvailFull)
limit = numAvailFull;
- for (; len2 < limit && data[len2] == data2[len2]; len2++);
-
- len2 -= len;
- if (len2 >= 3)
+ len2 += 2;
+ if (len2 <= limit)
+ if (data[len2 - 2] == data2[len2 - 2])
+ if (data[len2 - 1] == data2[len2 - 1])
{
unsigned state2 = kRepNextStates[state];
unsigned posState2 = (position + len) & p->pbMask;
- price +=
- p->repLenEnc.prices[posState][(size_t)len - 2]
+ price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
+ GET_PRICE_0(p->isMatch[state2][posState2])
+ LitEnc_Matched_GetPrice(LIT_PROBS(position + len, data[(size_t)len - 1]),
data[len], data2[len], p->ProbPrices);
@@ -1570,31 +1683,40 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price += GetPrice_Rep_0(p, state2, posState2);
+
+ for (; len2 < limit && data[len2] == data2[len2]; len2++)
+ {}
+
+ len2 -= len;
+ // if (len2 >= 3)
+ {
{
unsigned offset = cur + len + len2;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
- unsigned price2;
+ UInt32 price2;
COptimal *opt;
len2--;
// price2 = price + GetPrice_Len_Rep_0(p, len2, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len2 - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len2);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
+ opt->len = (UInt32)len2;
opt->extra = (CExtra)(len + 1);
- opt->dist = repIndex;
+ opt->dist = (UInt32)repIndex;
}
}
// while (len2 >= 3);
}
}
+ }
}
}
}
@@ -1606,17 +1728,23 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
{
newLen = numAvail;
for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
- matches[numPairs] = newLen;
+ matches[numPairs] = (UInt32)newLen;
numPairs += 2;
}
+ // startLen = 2; /* speed optimization */
+
if (newLen >= startLen)
{
UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[state]);
UInt32 dist;
unsigned offs, posSlot, len;
- while (last < cur + newLen)
- p->opt[++last].price = kInfinityPrice;
+
+ {
+ unsigned offset = cur + newLen;
+ if (last < offset)
+ last = offset;
+ }
offs = 0;
while (startLen > matches[offs])
@@ -1628,27 +1756,29 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
for (len = /*2*/ startLen; ; len++)
{
- UInt32 price = normalMatchPrice + p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
+ UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc, posState, len);
{
COptimal *opt;
- unsigned lenToPosState = len - 2; lenToPosState = GetLenToPosState2(lenToPosState);
+ unsigned lenNorm = len - 2;
+ lenNorm = GetLenToPosState2(lenNorm);
if (dist < kNumFullDistances)
- price += p->distancesPrices[lenToPosState][dist & (kNumFullDistances - 1)];
+ price += p->distancesPrices[lenNorm][dist & (kNumFullDistances - 1)];
else
- price += p->posSlotPrices[lenToPosState][posSlot] + p->alignPrices[dist & kAlignMask];
+ price += p->posSlotPrices[lenNorm][posSlot] + p->alignPrices[dist & kAlignMask];
opt = &p->opt[cur + len];
if (price < opt->price)
{
opt->price = price;
- opt->len = len;
+ opt->len = (UInt32)len;
opt->dist = dist + LZMA_NUM_REPS;
opt->extra = 0;
}
}
- if (/*_maxMode && */ len == matches[offs])
+ if (len == matches[offs])
{
+ // if (p->_maxMode) {
// MATCH : LIT : REP_0
const Byte *data2 = data - dist - 1;
@@ -1657,11 +1787,17 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
if (limit > numAvailFull)
limit = numAvailFull;
- for (; len2 < limit && data[len2] == data2[len2]; len2++);
+ len2 += 2;
+ if (len2 <= limit)
+ if (data[len2 - 2] == data2[len2 - 2])
+ if (data[len2 - 1] == data2[len2 - 1])
+ {
+ for (; len2 < limit && data[len2] == data2[len2]; len2++)
+ {}
len2 -= len;
- if (len2 >= 3)
+ // if (len2 >= 3)
{
unsigned state2 = kMatchNextStates[state];
unsigned posState2 = (position + len) & p->pbMask;
@@ -1677,27 +1813,30 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
price += GetPrice_Rep_0(p, state2, posState2);
offset = cur + len + len2;
- while (last < offset)
- p->opt[++last].price = kInfinityPrice;
+
+ if (last < offset)
+ last = offset;
// do
{
UInt32 price2;
COptimal *opt;
len2--;
// price2 = price + GetPrice_Len_Rep_0(p, len2, state2, posState2);
- price2 = price + p->repLenEnc.prices[posState2][len2 - LZMA_MATCH_LEN_MIN];
+ price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState2, len2);
opt = &p->opt[offset];
// offset--;
if (price2 < opt->price)
{
opt->price = price2;
- opt->len = len2;
+ opt->len = (UInt32)len2;
opt->extra = (CExtra)(len + 1);
opt->dist = dist + LZMA_NUM_REPS;
}
}
// while (len2 >= 3);
}
+
+ }
offs += 2;
if (offs == numPairs)
@@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32 position)
}
}
}
+
+ do
+ p->opt[last].price = kInfinityPrice;
+ while (--last);
+
+ return Backward(p, cur);
}
@@ -1735,6 +1880,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
p->backRes = MARK_LIT;
if (numAvail < 2)
return 1;
+ // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
if (numAvail > LZMA_MATCH_LEN_MAX)
numAvail = LZMA_MATCH_LEN_MAX;
data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
@@ -1746,10 +1892,11 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
const Byte *data2 = data - p->reps[i];
if (data[0] != data2[0] || data[1] != data2[1])
continue;
- for (len = 2; len < numAvail && data[len] == data2[len]; len++);
+ for (len = 2; len < numAvail && data[len] == data2[len]; len++)
+ {}
if (len >= p->numFastBytes)
{
- p->backRes = i;
+ p->backRes = (UInt32)i;
MOVE_POS(p, len - 1)
return len;
}
@@ -1793,7 +1940,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
|| (repLen + 2 >= mainLen && mainDist >= (1 << 9))
|| (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
{
- p->backRes = repIndex;
+ p->backRes = (UInt32)repIndex;
MOVE_POS(p, repLen - 1)
return repLen;
}
@@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p, UInt32 nowPos)
}
-
-static void FillAlignPrices(CLzmaEnc *p)
+MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
{
unsigned i;
const CProbPrice *ProbPrices = p->ProbPrices;
const CLzmaProb *probs = p->posAlignEncoder;
- p->alignPriceCount = 0;
+ // p->alignPriceCount = 0;
for (i = 0; i < kAlignTableSize / 2; i++)
{
UInt32 price = 0;
- unsigned symbol = i;
+ unsigned sym = i;
unsigned m = 1;
unsigned bit;
UInt32 prob;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m << 1) + bit;
prob = probs[m];
p->alignPrices[i ] = price + GET_PRICEa_0(prob);
p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
@@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
}
-static void FillDistancesPrices(CLzmaEnc *p)
+MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
{
+ // int y; for (y = 0; y < 100; y++) {
+
UInt32 tempPrices[kNumFullDistances];
- unsigned i, lenToPosState;
+ unsigned i, lps;
const CProbPrice *ProbPrices = p->ProbPrices;
p->matchPriceCount = 0;
- for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
+ for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
{
unsigned posSlot = GetPosSlot1(i);
- unsigned footerBits = ((posSlot >> 1) - 1);
+ unsigned footerBits = (posSlot >> 1) - 1;
unsigned base = ((2 | (posSlot & 1)) << footerBits);
+ const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
// tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base, footerBits, i - base, p->ProbPrices);
-
- const CLzmaProb *probs = p->posEncoders + base;
UInt32 price = 0;
unsigned m = 1;
- unsigned symbol = i - base;
+ unsigned sym = i;
+ unsigned offset = (unsigned)1 << footerBits;
+ base += i;
+
+ if (footerBits)
do
{
- unsigned bit = symbol & 1;
- symbol >>= 1;
+ unsigned bit = sym & 1;
+ sym >>= 1;
price += GET_PRICEa(probs[m], bit);
m = (m << 1) + bit;
}
while (--footerBits);
- tempPrices[i] = price;
+
+ {
+ unsigned prob = probs[m];
+ tempPrices[base ] = price + GET_PRICEa_0(prob);
+ tempPrices[base + offset] = price + GET_PRICEa_1(prob);
+ }
}
- for (lenToPosState = 0; lenToPosState < kNumLenToPosStates; lenToPosState++)
+ for (lps = 0; lps < kNumLenToPosStates; lps++)
{
- unsigned posSlot;
- const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
- UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
- unsigned distTableSize = p->distTableSize;
- const CLzmaProb *probs = encoder;
- for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
+ unsigned slot;
+ unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
+ UInt32 *posSlotPrices = p->posSlotPrices[lps];
+ const CLzmaProb *probs = p->posSlotEncoder[lps];
+
+ for (slot = 0; slot < distTableSize2; slot++)
{
- // posSlotPrices[posSlot] = RcTree_GetPrice(encoder, kNumPosSlotBits, posSlot, p->ProbPrices);
- UInt32 price = 0;
+ // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits, slot, p->ProbPrices);
+ UInt32 price;
unsigned bit;
- unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
- UInt32 prob;
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol], bit);
- prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
- posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
- posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
+ unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
+ unsigned prob;
+ bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
+ prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
+ posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
+ posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
}
- for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
- posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
-
+
{
- UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
+ UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
+ for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
{
- distancesPrices[0] = posSlotPrices[0];
- distancesPrices[1] = posSlotPrices[1];
- distancesPrices[2] = posSlotPrices[2];
- distancesPrices[3] = posSlotPrices[3];
+ posSlotPrices[(size_t)slot * 2 ] += delta;
+ posSlotPrices[(size_t)slot * 2 + 1] += delta;
+ delta += ((UInt32)1 << kNumBitPriceShiftBits);
}
+ }
+
+ {
+ UInt32 *dp = p->distancesPrices[lps];
+
+ dp[0] = posSlotPrices[0];
+ dp[1] = posSlotPrices[1];
+ dp[2] = posSlotPrices[2];
+ dp[3] = posSlotPrices[3];
+
for (i = 4; i < kNumFullDistances; i += 2)
{
UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
- distancesPrices[i ] = slotPrice + tempPrices[i];
- distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
+ dp[i ] = slotPrice + tempPrices[i];
+ dp[i + 1] = slotPrice + tempPrices[i + 1];
}
}
}
+ // }
}
@@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (len != 1)
{
LenEnc_Encode(&p->repLenProbs, &p->rc, len - LZMA_MATCH_LEN_MIN, posState);
- if (!p->fastMode)
- if (--p->repLenEnc.counters[posState] == 0)
- LenPriceEnc_UpdateTable(&p->repLenEnc, posState, &p->repLenProbs, p->ProbPrices);
-
+ --p->repLenEncCounter;
p->state = kRepNextStates[p->state];
}
}
@@ -2260,9 +2422,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
p->state = kMatchNextStates[p->state];
LenEnc_Encode(&p->lenProbs, &p->rc, len - LZMA_MATCH_LEN_MIN, posState);
- if (!p->fastMode)
- if (--p->lenEnc.counters[posState] == 0)
- LenPriceEnc_UpdateTable(&p->lenEnc, posState, &p->lenProbs, p->ProbPrices);
+ // --p->lenEnc.counter;
dist -= LZMA_NUM_REPS;
p->reps[3] = p->reps[2];
@@ -2274,17 +2434,17 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
GetPosSlot(dist, posSlot);
// RcTree_Encode_PosSlot(&p->rc, p->posSlotEncoder[GetLenToPosState(len)], posSlot);
{
- UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
+ UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
range = p->rc.range;
probs = p->posSlotEncoder[GetLenToPosState(len)];
do
{
- CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
- UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
- symbol <<= 1;
+ CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
+ UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
+ sym <<= 1;
RC_BIT(&p->rc, prob, bit);
}
- while (symbol < (1 << kNumPosSlotBits * 2));
+ while (sym < (1 << kNumPosSlotBits * 2));
p->rc.range = range;
}
@@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (dist < kNumFullDistances)
{
unsigned base = ((2 | (posSlot & 1)) << footerBits);
- RcTree_ReverseEncode(&p->rc, p->posEncoders + base, footerBits, dist - base);
+ RcTree_ReverseEncode(&p->rc, p->posEncoders + base, footerBits, (unsigned)(dist /* - base */));
}
else
{
@@ -2331,14 +2491,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
bit = dist & 1; dist >>= 1; RC_BIT(&p->rc, p->posAlignEncoder + m, bit); m = (m << 1) + bit;
bit = dist & 1; RC_BIT(&p->rc, p->posAlignEncoder + m, bit);
p->rc.range = range;
- p->alignPriceCount++;
+ // p->alignPriceCount++;
}
}
}
}
}
- nowPos32 += len;
+ nowPos32 += (UInt32)len;
p->additionalOffset -= len;
if (p->additionalOffset == 0)
@@ -2347,10 +2507,27 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, UInt32 maxPackSize, UInt32 maxUnpa
if (!p->fastMode)
{
- if (p->matchPriceCount >= (1 << 7))
+ /*
+ if (p->alignPriceCount >= 16) // kAlignTableSize
+ FillAlignPrices(p);
+ if (p->matchPriceCount >= 128)
FillDistancesPrices(p);
- if (p->alignPriceCount >= kAlignTableSize)
+ if (p->lenEnc.counter <= 0)
+ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
+ */
+ if (p->matchPriceCount >= 64)
+ {
FillAlignPrices(p);
+ // { int y; for (y = 0; y < 100; y++) {
+ FillDistancesPrices(p);
+ // }}
+ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
+ }
+ if (p->repLenEncCounter <= 0)
+ {
+ p->repLenEncCounter = REP_LEN_COUNT;
+ LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs, p->ProbPrices);
+ }
}
if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
@@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
p->optEnd = 0;
p->optCur = 0;
+
+ {
+ for (i = 0; i < kNumOpts; i++)
+ p->opt[i].price = kInfinityPrice;
+ }
+
p->additionalOffset = 0;
p->pbMask = (1 << p->pb) - 1;
p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
}
+
void LzmaEnc_InitPrices(CLzmaEnc *p)
{
if (!p->fastMode)
@@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
p->lenEnc.tableSize =
p->repLenEnc.tableSize =
p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
+
+ p->repLenEncCounter = REP_LEN_COUNT;
+
LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs, p->ProbPrices);
LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs, p->ProbPrices);
}
@@ -2583,7 +2770,7 @@ typedef struct
ISeqOutStream vt;
Byte *data;
SizeT rem;
- Bool overflow;
+ BoolInt overflow;
} CLzmaEnc_SeqOutStreamBuf;
static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void *data, size_t size)
@@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle pp)
}
-SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
+SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32 *unpackSize)
{
CLzmaEnc *p = (CLzmaEnc *)pp;
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
index d7426d38a4..04f9d98e7d 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
@@ -1,6 +1,28 @@
HISTORY of the LZMA SDK
-----------------------
+19.00 2019-02-21
+-------------------------
+- Encryption strength for 7z archives was increased:
+ the size of random initialization vector was increased from 64-bit to 128-bit,
+ and the pseudo-random number generator was improved.
+- The bug in 7zIn.c code was fixed.
+
+
+18.06 2018-12-30
+-------------------------
+- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
+ and there are minor changes in compression ratio.
+- Some bugs were fixed.
+- The bug in 7-Zip 18.02-18.05 was fixed:
+ There was memory leak in multithreading xz decoder - XzDecMt_Decode(),
+ if xz stream contains only one block.
+- The changes for MSVS compiler makefiles:
+ - the makefiles now use "PLATFORM" macroname with values (x64, x86, arm64)
+ instead of "CPU" macroname with values (AMD64, ARM64).
+ - the makefiles by default now use static version of the run-time library.
+
+
18.05 2018-04-30
-------------------------
- The speed for LZMA/LZMA2 compressing was increased
diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
index a3deb20943..e17cd4fe1a 100644
--- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
+++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
@@ -1,4 +1,4 @@
-LZMA SDK 18.05
+LZMA SDK 19.00
--------------
LZMA SDK provides the documentation, samples, header files,
--
2.27.0.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* 回复: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
2020-12-15 2:06 Wei Liu
@ 2020-12-17 1:30 ` gaoliming
2020-12-20 13:33 ` Liu, WeiX C
0 siblings, 1 reply; 7+ messages in thread
From: gaoliming @ 2020-12-17 1:30 UTC (permalink / raw)
To: 'Wei Liu', devel; +Cc: 'Laszlo Ersek'
Wei:
Can you list the unit test for this patch? Which tool chain and arch have
been verified?
Because LZMA is well used, I request to verify it on VS, GCC, XCODE and
CLANG tool chain for the different ARCHs.
Thanks
Liming
> -----邮件原件-----
> 发件人: Wei Liu <weix.c.liu@intel.com>
> 发送时间: 2020年12月15日 10:07
> 收件人: devel@edk2.groups.io
> 抄送: Wei Liu <weix.c.liu@intel.com>; Liming Gao
> <gaoliming@byosoft.com.cn>; Laszlo Ersek <lersek@redhat.com>
> 主题: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00
> version
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
> New formal release in https://www.7-zip.org/sdk.html is 19.00.
>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Signed-off-by: Wei Liu <weix.c.liu@intel.com>
> ---
> BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4
> ++--
> BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5
> +++--
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9
> +++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 164
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++++++++++++++++++++++++++++++++++++++--------------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14
> +++++++-------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32
> ++++++++++++++++----------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 733
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
++++++++++++++++++++++++++--------------------------------------------------
----
>
----------------------------------------------------------------------------
-----------------
>
----------------------------------------------------------------------------
-----------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 22
> ++++++++++++++++++++++
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
> 12 files changed, 710 insertions(+), 408 deletions(-)
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> index 7cf967a774..b2338e2598 100644
> --- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> +++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> @@ -1,3 +1,3 @@
> -LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
>
> -was placed in the public domain on 2018-04-30. It was
>
> +LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
>
> +was placed in the public domain on 2019-02-21. It was
>
> released on the http://www.7-zip.org/sdk.html website.
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> index bebdb9aa84..bf4706ee23 100644
> --- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> +++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> @@ -1,9 +1,9 @@
> /** @file
>
> LZMA Compress/Decompress tool (LzmaCompress)
>
>
>
> - Based on LZMA SDK 18.05:
>
> + Based on LZMA SDK 19.00:
>
> LzmaUtil.c -- Test application for LZMA compression
>
> - 2018-04-30 : Igor Pavlov : Public domain
>
> + 2019-02-21 : Igor Pavlov : Public domain
>
>
>
> Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
>
> @@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate
> memory";
> const char *kDataErrorMessage = "Data error";
>
> const char *kInvalidParamValMessage = "Invalid parameter value";
>
>
>
> -static Bool mQuietMode = False;
>
> +static BoolInt mQuietMode = False;
>
> static CONVERTER_TYPE mConType = NoConverter;
>
>
>
> UINT64 mDictionarySize = 28;
>
> @@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
> CFileOutStream outStream;
>
> int res;
>
> int encodeMode = 0;
>
> - Bool modeWasSet = False;
>
> + BoolInt modeWasSet = False;
>
> const char *inputFile = NULL;
>
> const char *outputFile = "file.tmp";
>
> int param;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> index 4977cdaa66..593f5aa259 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> @@ -1,5 +1,5 @@
> /* 7zTypes.h -- Basic types
>
> -2017-07-17 : Igor Pavlov : Public domain */
>
> +2018-08-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __7Z_TYPES_H
>
> #define __7Z_TYPES_H
>
> @@ -103,7 +103,8 @@ typedef UInt32 SizeT;
> typedef size_t SizeT;
>
> #endif
>
>
>
> -typedef int Bool;
>
> +typedef int BoolInt;
>
> +/* typedef BoolInt Bool; */
>
> #define True 1
>
> #define False 0
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> index ed3aa94270..0074c64be9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> @@ -1,7 +1,7 @@
> -#define MY_VER_MAJOR 18
>
> -#define MY_VER_MINOR 05
>
> +#define MY_VER_MAJOR 19
>
> +#define MY_VER_MINOR 00
>
> #define MY_VER_BUILD 0
>
> -#define MY_VERSION_NUMBERS "18.05"
>
> +#define MY_VERSION_NUMBERS "19.00"
>
> #define MY_VERSION MY_VERSION_NUMBERS
>
>
>
> #ifdef MY_CPU_NAME
>
> @@ -10,7 +10,7 @@
> #define MY_VERSION_CPU MY_VERSION
>
> #endif
>
>
>
> -#define MY_DATE "2018-04-30"
>
> +#define MY_DATE "2019-02-21"
>
> #undef MY_COPYRIGHT
>
> #undef MY_VERSION_COPYRIGHT_DATE
>
> #define MY_AUTHOR_NAME "Igor Pavlov"
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> index 7fb27282c7..5f74c1c0cb 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> @@ -1,5 +1,5 @@
> /* CpuArch.h -- CPU specific code
>
> -2017-09-04 : Igor Pavlov : Public domain */
>
> +2018-02-18 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __CPU_ARCH_H
>
> #define __CPU_ARCH_H
>
> @@ -318,15 +318,16 @@ enum
>
>
> void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32
*d);
>
>
>
> -Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> +BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> int x86cpuid_GetFirm(const Cx86cpuid *p);
>
>
>
> #define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) &
0xF))
>
> #define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) &
> 0xF))
>
> #define x86cpuid_GetStepping(ver) (ver & 0xF)
>
>
>
> -Bool CPU_Is_InOrder();
>
> -Bool CPU_Is_Aes_Supported();
>
> +BoolInt CPU_Is_InOrder();
>
> +BoolInt CPU_Is_Aes_Supported();
>
> +BoolInt CPU_IsSupported_PageGB();
>
>
>
> #endif
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> index 6ea82a9b53..4eefc17dd2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> @@ -1,5 +1,5 @@
> /* LzFind.c -- Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-07-08 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -138,7 +138,7 @@ static void
> MatchFinder_SetDefaultSettings(CMatchFinder *p)
>
>
> void MatchFinder_Construct(CMatchFinder *p)
>
> {
>
> - UInt32 i;
>
> + unsigned i;
>
> p->bufferBase = NULL;
>
> p->directInput = 0;
>
> p->hash = NULL;
>
> @@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
>
>
> for (i = 0; i < 256; i++)
>
> {
>
> - UInt32 r = i;
>
> + UInt32 r = (UInt32)i;
>
> unsigned j;
>
> for (j = 0; j < 8; j++)
>
> r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
>
> @@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
> MatchFinder_ReduceOffsets(p, subValue);
>
> }
>
>
>
> +
>
> +MY_NO_INLINE
>
> static void MatchFinder_CheckLimits(CMatchFinder *p)
>
> {
>
> if (p->pos == kMaxValForNormalize)
>
> @@ -379,10 +381,16 @@ static void
> MatchFinder_CheckLimits(CMatchFinder *p)
> MatchFinder_SetLimits(p);
>
> }
>
>
>
> -static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> +
>
> +/*
>
> + (lenLimit > maxLen)
>
> +*/
>
> +MY_FORCE_INLINE
>
> +static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> - UInt32 *distances, UInt32 maxLen)
>
> + UInt32 *distances, unsigned maxLen)
>
> {
>
> + /*
>
> son[_cyclicBufferPos] = curMatch;
>
> for (;;)
>
> {
>
> @@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos,
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> return distances;
>
> @@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32
> lenLimit, UInt32 curMatch, UInt32 pos,
> }
>
> }
>
> }
>
> + */
>
> +
>
> + const Byte *lim = cur + lenLimit;
>
> + son[_cyclicBufferPos] = curMatch;
>
> + do
>
> + {
>
> + UInt32 delta = pos - curMatch;
>
> + if (delta >= _cyclicBufferSize)
>
> + break;
>
> + {
>
> + ptrdiff_t diff;
>
> + curMatch = son[_cyclicBufferPos - delta + ((delta >
_cyclicBufferPos) ?
> _cyclicBufferSize : 0)];
>
> + diff = (ptrdiff_t)0 - delta;
>
> + if (cur[maxLen] == cur[maxLen + diff])
>
> + {
>
> + const Byte *c = cur;
>
> + while (*c == c[diff])
>
> + {
>
> + if (++c == lim)
>
> + {
>
> + distances[0] = (UInt32)(lim - cur);
>
> + distances[1] = delta - 1;
>
> + return distances + 2;
>
> + }
>
> + }
>
> + {
>
> + unsigned len = (unsigned)(c - cur);
>
> + if (maxLen < len)
>
> + {
>
> + maxLen = len;
>
> + distances[0] = (UInt32)len;
>
> + distances[1] = delta - 1;
>
> + distances += 2;
>
> + }
>
> + }
>
> + }
>
> + }
>
> + }
>
> + while (--cutValue);
>
> +
>
> + return distances;
>
> }
>
>
>
> +
>
> +MY_FORCE_INLINE
>
> UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> UInt32 *distances, UInt32 maxLen)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> return distances;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = pair[0];
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const Byt
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = (UInt32)len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> + *ptr1 = pair0;
>
> *ptr0 = pair[1];
>
> return distances;
>
> }
>
> @@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const
> return;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> if (pb[len] == cur[len])
>
> {
>
> while (++len != lenLimit)
>
> @@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const
> p->buffer++; \
>
> if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
>
>
>
> -#define MOVE_POS_RET MOVE_POS return offset;
>
> +#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
>
>
>
> static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
>
>
>
> #define GET_MATCHES_HEADER2(minLen, ret_op) \
>
> - UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> - lenLimit = p->lenLimit; { if (lenLimit < minLen) {
MatchFinder_MovePos(p);
> ret_op; }} \
>
> + unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> + lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen)
> { MatchFinder_MovePos(p); ret_op; }} \
>
> cur = p->buffer;
>
>
>
> #define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen,
> return 0)
>
> @@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder
> *p) { MOVE_POS; }
> #define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos,
> p->cyclicBufferSize, p->cutValue
>
>
>
> #define GET_MATCHES_FOOTER(offset, maxLen) \
>
> - offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
>
> - distances + offset, maxLen) - distances); MOVE_POS_RET;
>
> + offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch,
> MF_PARAMS(p), \
>
> + distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
>
>
>
> #define SKIP_FOOTER \
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
> MOVE_POS;
>
>
>
> #define UPDATE_maxLen { \
>
> ptrdiff_t diff = (ptrdiff_t)0 - d2; \
>
> const Byte *c = cur + maxLen; \
>
> const Byte *lim = cur + lenLimit; \
>
> for (; c != lim; c++) if (*(c + diff) != *c) break; \
>
> - maxLen = (UInt32)(c - cur); }
>
> + maxLen = (unsigned)(c - cur); }
>
>
>
> static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(2)
>
> HASH2_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -561,7 +615,7 @@ static UInt32
> Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -572,7 +626,8 @@ UInt32
> Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, d2, maxLen, offset, pos;
>
> + UInt32 h2, d2, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(3)
>
>
>
> @@ -594,12 +649,12 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> UPDATE_maxLen
>
> - distances[0] = maxLen;
>
> + distances[0] = (UInt32)maxLen;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -609,7 +664,8 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -618,12 +674,12 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
>
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -632,7 +688,8 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -648,10 +705,10 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -674,13 +731,13 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -741,7 +798,8 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -750,12 +808,11 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> -
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -764,7 +821,8 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -780,7 +838,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -791,7 +849,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (maxLen < 3)
>
> maxLen = 3;
>
>
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances + offset, maxLen) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -808,13 +866,13 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -877,12 +935,12 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> p->hash[hv] = p->pos;
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances, 2) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> SKIP_FOOTER
>
> @@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> @@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> index 2563824fcd..df32146f92 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> @@ -1,5 +1,5 @@
> /* LzFindMt.c -- multithreaded Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-12-29 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -232,38 +232,57 @@ static void
> MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
>
>
> #define kEmptyHashValue 0
>
>
>
> -/* #define MFMT_GM_INLINE */
>
> +#define MFMT_GM_INLINE
>
>
>
> #ifdef MFMT_GM_INLINE
>
>
>
> -#define NO_INLINE MY_FAST_CALL
>
> +/*
>
> + we use size_t for _cyclicBufferPos instead of UInt32
>
> + to eliminate "movsx" BUG in old MSVC x64 compiler.
>
> +*/
>
>
>
> -static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> - UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> - UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit,
> UInt32 size, UInt32 *posRes)
>
> +MY_NO_INLINE
>
> +static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte
> *cur, CLzRef *son,
>
> + size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> + UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32
> *limit, UInt32 size, UInt32 *posRes)
>
> {
>
> do
>
> {
>
> - UInt32 *distances = _distances + 1;
>
> - UInt32 curMatch = pos - *hash++;
>
> + UInt32 *_distances = ++distances;
>
> + UInt32 delta = *hash++;
>
>
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> UInt32 cutValue = _cutValue;
>
> - UInt32 maxLen = _maxLen;
>
> - for (;;)
>
> + unsigned maxLen = (unsigned)_maxLen;
>
> +
>
> + /*
>
> + if (size > 1)
>
> {
>
> - UInt32 delta = pos - curMatch;
>
> - if (cutValue-- == 0 || delta >= _cyclicBufferSize)
>
> + UInt32 delta = *hash;
>
> + if (delta < _cyclicBufferSize)
>
> {
>
> - *ptr0 = *ptr1 = kEmptyHashValue;
>
> - break;
>
> + UInt32 cyc1 = _cyclicBufferPos + 1;
>
> + CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ?
> _cyclicBufferSize : 0)) << 1);
>
> + Byte b = *(cur + 1 - delta);
>
> + _distances[0] = pair[0];
>
> + _distances[1] = b;
>
> }
>
> + }
>
> + */
>
> + if (cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + }
>
> + else
>
> + for(;;)
>
> + {
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta +
> ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = *pair;
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32
> lenLimit, UInt32 pos, const Byte *
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> - *ptr0 = pair[1];
>
> + UInt32 pair1 = pair[1];
>
> + *ptr1 = pair0;
>
> + *ptr0 = pair1;
>
> break;
>
> }
>
> }
>
> }
>
> - if (pb[len] < cur[len])
>
> - {
>
> - *ptr1 = curMatch;
>
> - ptr1 = pair + 1;
>
> - curMatch = *ptr1;
>
> - len1 = len;
>
> - }
>
> - else
>
> {
>
> - *ptr0 = curMatch;
>
> - ptr0 = pair;
>
> - curMatch = *ptr0;
>
> - len0 = len;
>
> + UInt32 curMatch = pos - delta;
>
> + // delta = pos - *pair;
>
> + // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >>
31];
>
> + if (pb[len] < cur[len])
>
> + {
>
> + delta = pos - pair[1];
>
> + *ptr1 = curMatch;
>
> + ptr1 = pair + 1;
>
> + len1 = len;
>
> + }
>
> + else
>
> + {
>
> + delta = pos - *pair;
>
> + *ptr0 = curMatch;
>
> + ptr0 = pair;
>
> + len0 = len;
>
> + }
>
> }
>
> }
>
> + if (--cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + break;
>
> + }
>
> }
>
> pos++;
>
> _cyclicBufferPos++;
>
> cur++;
>
> {
>
> UInt32 num = (UInt32)(distances - _distances);
>
> - *_distances = num - 1;
>
> - _distances += num;
>
> - limit -= num;
>
> + _distances[-1] = num;
>
> }
>
> }
>
> - while (limit > 0 && --size != 0);
>
> + while (distances < limit && --size != 0);
>
> *posRes = pos;
>
> - return limit;
>
> + return distances;
>
> }
>
>
>
> #endif
>
>
>
> +
>
> +
>
> static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
>
> {
>
> UInt32 numProcessed = 0;
>
> UInt32 curPos = 2;
>
> - UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
>
> + UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
>
>
>
> distances[1] = p->hashNumAvail;
>
>
>
> @@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p,
> UInt32 *distances)
> #else
>
> {
>
> UInt32 posRes;
>
> - curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer,
p->son,
> cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> - distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
>
> + curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer,
> p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> + distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos,
>
> + distances + limit,
>
> + size, &posRes) - distances);
>
> p->hashBufPos += posRes - pos;
>
> cyclicBufferPos += posRes - pos;
>
> p->buffer += posRes - pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> index 3d86c788f3..fdd17008c2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> @@ -1,5 +1,5 @@
> /* LzFindMt.h -- multithreaded Match finder for LZ algorithms
>
> -2017-04-03 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __LZ_FIND_MT_H
>
> #define __LZ_FIND_MT_H
>
> @@ -19,10 +19,10 @@ EXTERN_C_BEGIN
>
>
> typedef struct _CMtSync
>
> {
>
> - Bool wasCreated;
>
> - Bool needStart;
>
> - Bool exit;
>
> - Bool stopWriting;
>
> + BoolInt wasCreated;
>
> + BoolInt needStart;
>
> + BoolInt exit;
>
> + BoolInt stopWriting;
>
>
>
> CThread thread;
>
> CAutoResetEvent canStart;
>
> @@ -30,8 +30,8 @@ typedef struct _CMtSync
> CAutoResetEvent wasStopped;
>
> CSemaphore freeSemaphore;
>
> CSemaphore filledSemaphore;
>
> - Bool csWasInitialized;
>
> - Bool csWasEntered;
>
> + BoolInt csWasInitialized;
>
> + BoolInt csWasEntered;
>
> CCriticalSection cs;
>
> UInt32 numProcessedBlocks;
>
> } CMtSync;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> index 962b94bb63..4d1576419f 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> @@ -1,13 +1,13 @@
> /* LzmaDec.c -- LZMA Decoder
>
> -2018-02-28 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> +#include <string.h>
>
> +
>
> /* #include "CpuArch.h" */
>
> #include "LzmaDec.h"
>
>
>
> -#include <string.h>
>
> -
>
> #define kNumTopBits 24
>
> #define kTopValue ((UInt32)1 << kNumTopBits)
>
>
>
> @@ -19,7 +19,7 @@
>
>
> #define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code <<
8)
> | (*buf++); }
>
>
>
> -#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt +
> ((kBitModelTotal - ttt) >> kNumMoveBits));
>
> #define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt
> - (ttt >> kNumMoveBits));
>
> #define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
>
> @@ -66,7 +66,7 @@
>
>
> #define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit)
> return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
>
>
>
> -#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0_CHECK range = bound;
>
> #define UPDATE_1_CHECK range -= bound; code -= bound;
>
> #define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
>
> @@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> curLen = ((rem < len) ? (unsigned)rem : len);
>
> pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
>
>
>
> - processedPos += curLen;
>
> + processedPos += (UInt32)curLen;
>
>
>
> len -= curLen;
>
> if (curLen <= dicBufSize - pos)
>
> @@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> Byte *dest = dic + dicPos;
>
> ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
>
> const Byte *lim = dest + curLen;
>
> - dicPos += curLen;
>
> + dicPos += (SizeT)curLen;
>
> do
>
> *(dest) = (Byte)*(dest + src);
>
> while (++dest != lim);
>
> @@ -572,14 +572,14 @@ int MY_FAST_CALL
> LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
> p->buf = buf;
>
> p->range = range;
>
> p->code = code;
>
> - p->remainLen = len;
>
> + p->remainLen = (UInt32)len;
>
> p->dicPos = dicPos;
>
> p->processedPos = processedPos;
>
> p->reps[0] = rep0;
>
> p->reps[1] = rep1;
>
> p->reps[2] = rep2;
>
> p->reps[3] = rep3;
>
> - p->state = state;
>
> + p->state = (UInt32)state;
>
>
>
> return SZ_OK;
>
> }
>
> @@ -601,8 +601,8 @@ static void MY_FAST_CALL
> LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
> if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
>
> p->checkDicSize = p->prop.dicSize;
>
>
>
> - p->processedPos += len;
>
> - p->remainLen -= len;
>
> + p->processedPos += (UInt32)len;
>
> + p->remainLen -= (UInt32)len;
>
> while (len != 0)
>
> {
>
> len--;
>
> @@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const
> CLzmaDec *p, const Byte *buf, SizeT inS
> }
>
>
>
>
>
> -void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
>
> +void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt
> initState)
>
> {
>
> p->remainLen = kMatchSpecLenStart + 1;
>
> p->tempBufSize = 0;
>
> @@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> p->tempBufSize = rem;
>
> if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
>
> {
>
> - int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
>
> + int dummyRes = LzmaDec_TryDummy(p, p->tempBuf,
> (SizeT)rem);
>
> if (dummyRes == DUMMY_ERROR)
>
> {
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> *status = LZMA_STATUS_NEEDS_MORE_INPUT;
>
> return SZ_OK;
>
> }
>
> @@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> return SZ_ERROR_FAIL; /* some internal error */
>
> lookAhead -= rem;
>
> }
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> src += lookAhead;
>
> - inSize -= lookAhead;
>
> + inSize -= (SizeT)lookAhead;
>
> p->tempBufSize = 0;
>
> }
>
> }
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> index e281716fee..14086fc4f9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> @@ -1,5 +1,5 @@
> /* LzmaEnc.c -- LZMA Encoder
>
> -2018-04-29 : Igor Pavlov : Public domain */
>
> +2019-01-10: Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
> #define kNumBitPriceShiftBits 4
>
> #define kBitPrice (1 << kNumBitPriceShiftBits)
>
>
>
> +#define REP_LEN_COUNT 64
>
> +
>
> void LzmaEncProps_Init(CLzmaEncProps *p)
>
> {
>
> p->level = 5;
>
> @@ -183,8 +185,11 @@ typedef struct
> } COptimal;
>
>
>
>
>
> -#define kNumOpts (1 << 12)
>
> -#define kPackReserve (1 + kNumOpts * 2)
>
> +// 18.06
>
> +#define kNumOpts (1 << 11)
>
> +#define kPackReserve (kNumOpts * 8)
>
> +// #define kNumOpts (1 << 12)
>
> +// #define kPackReserve (1 + kNumOpts * 2)
>
>
>
> #define kNumLenToPosStates 4
>
> #define kNumPosSlotBits 6
>
> @@ -236,10 +241,18 @@ typedef struct
> typedef struct
>
> {
>
> unsigned tableSize;
>
> - unsigned counters[LZMA_NUM_PB_STATES_MAX];
>
> UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
>
> + // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols *
> 2];
>
> + // UInt32 prices2[kLenNumSymbolsTotal];
>
> } CLenPriceEnc;
>
>
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
>
> +
>
> +/*
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) -
2) &
> (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >>
9)))
>
> +*/
>
>
>
> typedef struct
>
> {
>
> @@ -308,16 +321,18 @@ typedef struct
> unsigned lc, lp, pb;
>
> unsigned lclp;
>
>
>
> - Bool fastMode;
>
> - Bool writeEndMark;
>
> - Bool finished;
>
> - Bool multiThread;
>
> - Bool needInit;
>
> + BoolInt fastMode;
>
> + BoolInt writeEndMark;
>
> + BoolInt finished;
>
> + BoolInt multiThread;
>
> + BoolInt needInit;
>
> + // BoolInt _maxMode;
>
>
>
> UInt64 nowPos64;
>
>
>
> unsigned matchPriceCount;
>
> - unsigned alignPriceCount;
>
> + // unsigned alignPriceCount;
>
> + int repLenEncCounter;
>
>
>
> unsigned distTableSize;
>
>
>
> @@ -325,7 +340,7 @@ typedef struct
> SRes result;
>
>
>
> #ifndef _7ZIP_ST
>
> - Bool mtMode;
>
> + BoolInt mtMode;
>
> // begin of CMatchFinderMt is used in LZ thread
>
> CMatchFinderMt matchFinderMt;
>
> // end of CMatchFinderMt is used in BT and HASH threads
>
> @@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const
> CLzmaEncProps *props2)
> p->lp = props.lp;
>
> p->pb = props.pb;
>
> p->fastMode = (props.algo == 0);
>
> + // p->_maxMode = True;
>
> p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
>
> {
>
> unsigned numHashBytes = 4;
>
> @@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #ifdef _LZMA_ENC_USE_BRANCH
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> RC_BIT_PRE(p, prob) \
>
> - if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> + if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> else { (p)->low += newBound; range -= newBound; ttt -= ttt >>
> kNumMoveBits; } \
>
> *(prob) = (CLzmaProb)ttt; \
>
> RC_NORM(p) \
>
> @@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #else
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> UInt32 mask; \
>
> RC_BIT_PRE(p, prob) \
>
> - mask = 0 - (UInt32)symbol; \
>
> + mask = 0 - (UInt32)bit; \
>
> range &= mask; \
>
> mask &= newBound; \
>
> range -= mask; \
>
> (p)->low += mask; \
>
> - mask = (UInt32)symbol - 1; \
>
> + mask = (UInt32)bit - 1; \
>
> range += newBound & mask; \
>
> mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
>
> mask += ((1 << kNumMoveBits) - 1); \
>
> @@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p,
> CLzmaProb *prob)
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
>
> +static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
>
> {
>
> UInt32 range = p->range;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
>
> - CLzmaProb *prob = probs + (symbol >> 8);
>
> - UInt32 bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> + // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
>
> + CLzmaProb *prob = probs + (sym >> 8);
>
> + UInt32 bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte)
>
> +static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs,
> UInt32 sym, UInt32 matchByte)
>
> {
>
> UInt32 range = p->range;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> CLzmaProb *prob;
>
> UInt32 bit;
>
> matchByte <<= 1;
>
> - // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol
>>
> 8)), (symbol >> 7) & 1);
>
> - prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
>
> - bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >>
> 8)), (sym >> 7) & 1);
>
> + prob = probs + (offs + (matchByte & offs) + (sym >> 8));
>
> + bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> @@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> }
>
>
>
>
>
> -#define GET_PRICE(prob, symbol) \
>
> - p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICE(prob, bit) \
>
> + p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> -#define GET_PRICEa(prob, symbol) \
>
> - ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICEa(prob, bit) \
>
> + ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> #define GET_PRICE_0(prob) p->ProbPrices[(prob) >>
> kNumMoveReducingBits]
>
> #define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1))
>>
> kNumMoveReducingBits]
>
> @@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> #define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >>
> kNumMoveReducingBits]
>
>
>
>
>
> -static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol,
const
> CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const
> CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> - price += GET_PRICEa(probs[symbol], bit);
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> }
>
> - while (symbol >= 2);
>
> + while (sym >= 2);
>
> return price;
>
> }
>
>
>
>
>
> -static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> sym, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> matchByte <<= 1;
>
> - price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)],
> (symbol >> 7) & 1);
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)],
> (sym >> 7) & 1);
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> return price;
>
> }
>
>
>
>
>
> -static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, UInt32 symbol)
>
> +static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, unsigned sym)
>
> {
>
> UInt32 range = rc->range;
>
> unsigned m = 1;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - unsigned bit = symbol & 1;
>
> + unsigned bit = sym & 1;
>
> // RangeEnc_EncodeBit(rc, probs + m, bit);
>
> - symbol >>= 1;
>
> + sym >>= 1;
>
> RC_BIT(rc, probs + m, bit);
>
> m = (m << 1) | bit;
>
> }
>
> @@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
> p->high[i] = kProbInitValue;
>
> }
>
>
>
> -static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol,
> unsigned posState)
>
> +static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym,
> unsigned posState)
>
> {
>
> UInt32 range, ttt, newBound;
>
> CLzmaProb *probs = p->low;
>
> range = rc->range;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols)
>
> + if (sym >= kLenNumLowSymbols)
>
> {
>
> RC_BIT_1(rc, probs);
>
> probs += kLenNumLowSymbols;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols * 2)
>
> + if (sym >= kLenNumLowSymbols * 2)
>
> {
>
> RC_BIT_1(rc, probs);
>
> rc->range = range;
>
> - // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol -
> kLenNumLowSymbols * 2);
>
> - LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
>
> + // RcTree_Encode(rc, p->high, kLenNumHighBits, sym -
> kLenNumLowSymbols * 2);
>
> + LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
>
> return;
>
> }
>
> - symbol -= kLenNumLowSymbols;
>
> + sym -= kLenNumLowSymbols;
>
> }
>
>
>
> - // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, symbol);
>
> + // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, sym);
>
> {
>
> unsigned m;
>
> unsigned bit;
>
> RC_BIT_0(rc, probs);
>
> probs += (posState << (1 + kLenNumLowBits));
>
> - bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) +
bit;
>
> - bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) +
bit;
>
> - bit = symbol & 1; RC_BIT(rc, probs + m, bit);
>
> + bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
>
> + bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
>
> + bit = sym & 1; RC_BIT(rc, probs + m, bit);
>
> rc->range = range;
>
> }
>
> }
>
> @@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs,
> UInt32 startPrice, UInt32 *price
> }
>
>
>
>
>
> -MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
>
> - CLenPriceEnc *p, unsigned posState,
>
> +MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
>
> + CLenPriceEnc *p,
>
> + unsigned numPosStates,
>
> const CLenEnc *enc,
>
> const CProbPrice *ProbPrices)
>
> {
>
> - // int y; for (y = 0; y < 100; y++) {
>
> - UInt32 a;
>
> - unsigned i, numSymbols;
>
> + UInt32 b;
>
> +
>
> + {
>
> + unsigned prob = enc->low[0];
>
> + UInt32 a, c;
>
> + unsigned posState;
>
> + b = GET_PRICEa_1(prob);
>
> + a = GET_PRICEa_0(prob);
>
> + c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (posState = 0; posState < numPosStates; posState++)
>
> + {
>
> + UInt32 *prices = p->prices[posState];
>
> + const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> + SetPrices_3(probs, a, prices, ProbPrices);
>
> + SetPrices_3(probs + kLenNumLowSymbols, c, prices +
> kLenNumLowSymbols, ProbPrices);
>
> + }
>
> + }
>
>
>
> - UInt32 *prices = p->prices[posState];
>
> + /*
>
> {
>
> - const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> - SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
>
> + unsigned i;
>
> + UInt32 b;
>
> + a = GET_PRICEa_0(enc->low[0]);
>
> + for (i = 0; i < kLenNumLowSymbols; i++)
>
> + p->prices2[i] = a;
>
> a = GET_PRICEa_1(enc->low[0]);
>
> - SetPrices_3(probs + kLenNumLowSymbols, a +
> GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices +
> kLenNumLowSymbols, ProbPrices);
>
> + b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
>
> + p->prices2[i] = b;
>
> a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> }
>
> - numSymbols = p->tableSize;
>
> - p->counters[posState] = numSymbols;
>
> - for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
>
> + */
>
> +
>
> + // p->counter = numSymbols;
>
> + // p->counter = 64;
>
> +
>
> {
>
> - prices[i] = a +
>
> - // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> - LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2,
> ProbPrices);
>
> - /*
>
> - unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
>
> - UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> - UInt32 prob = enc->high[(1 << 7) + sym];
>
> - prices[i ] = price + GET_PRICEa_0(prob);
>
> - prices[i + 1] = price + GET_PRICEa_1(prob);
>
> - */
>
> - }
>
> - // }
>
> -}
>
> + unsigned i = p->tableSize;
>
> +
>
> + if (i > kLenNumLowSymbols * 2)
>
> + {
>
> + const CLzmaProb *probs = enc->high;
>
> + UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
>
> + i -= kLenNumLowSymbols * 2 - 1;
>
> + i >>= 1;
>
> + b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> + do
>
> + {
>
> + /*
>
> + p->prices2[i] = a +
>
> + // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> + LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
>
> + */
>
> + // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> + unsigned sym = --i + (1 << (kLenNumHighBits - 1));
>
> + UInt32 price = b;
>
> + do
>
> + {
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> + }
>
> + while (sym >= 2);
>
>
>
> -static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned
> numPosStates,
>
> - const CLenEnc *enc,
>
> - const CProbPrice *ProbPrices)
>
> -{
>
> - unsigned posState;
>
> - for (posState = 0; posState < numPosStates; posState++)
>
> - LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
>
> -}
>
> + {
>
> + unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits -
1))];
>
> + prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
>
> + prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
>
> + }
>
> + }
>
> + while (i);
>
>
>
> + {
>
> + unsigned posState;
>
> + size_t num = (p->tableSize - kLenNumLowSymbols * 2) *
> sizeof(p->prices[0][0]);
>
> + for (posState = 1; posState < numPosStates; posState++)
>
> + memcpy(p->prices[posState] + kLenNumLowSymbols * 2,
> p->prices[0] + kLenNumLowSymbols * 2, num);
>
> + }
>
> + }
>
> + }
>
> +}
>
>
>
> /*
>
> #ifdef SHOW_STAT
>
> @@ -917,7 +976,7 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc
> *p, unsigned numPosStates,
>
>
> #define MOVE_POS(p, num) { \
>
> p->additionalOffset += (num); \
>
> - p->matchFinder.Skip(p->matchFinderObj, (num)); }
>
> + p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
>
>
>
>
>
> static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
>
> @@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> const Byte *p2 = p1 + len;
>
> ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
>
> const Byte *lim = p1 + numAvail;
>
> - for (; p2 != lim && *p2 == p2[dif]; p2++);
>
> + for (; p2 != lim && *p2 == p2[dif]; p2++)
>
> + {}
>
> return (unsigned)(p2 - p1);
>
> }
>
> }
>
> @@ -977,7 +1037,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> + GET_PRICE_1(p->isRep[state]) \
>
> + GET_PRICE_0(p->isRepG0[state])
>
>
>
> -
>
> +MY_FORCE_INLINE
>
> static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex,
> size_t state, size_t posState)
>
> {
>
> UInt32 price;
>
> @@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p,
> unsigned cur)
> for (;;)
>
> {
>
> UInt32 dist = p->opt[cur].dist;
>
> - UInt32 len = p->opt[cur].len;
>
> - UInt32 extra = p->opt[cur].extra;
>
> + unsigned len = (unsigned)p->opt[cur].len;
>
> + unsigned extra = (unsigned)p->opt[cur].extra;
>
> cur -= len;
>
>
>
> if (extra)
>
> {
>
> wr--;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> cur -= extra;
>
> len = extra;
>
> if (extra == 1)
>
> @@ -1045,7 +1105,7 @@ static unsigned Backward(CLzmaEnc *p, unsigned
> cur)
>
>
> wr--;
>
> p->opt[wr].dist = dist;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> }
>
> }
>
>
>
> @@ -1102,7 +1162,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> repLens[i] = 0;
>
> continue;
>
> }
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> repLens[i] = len;
>
> if (len > repLens[repMaxIndex])
>
> repMaxIndex = i;
>
> @@ -1111,7 +1172,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (repLens[repMaxIndex] >= p->numFastBytes)
>
> {
>
> unsigned len;
>
> - p->backRes = repMaxIndex;
>
> + p->backRes = (UInt32)repMaxIndex;
>
> len = repLens[repMaxIndex];
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> @@ -1128,8 +1189,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
> +
>
> + last = repLens[repMaxIndex];
>
> + if (last <= mainLen)
>
> + last = mainLen;
>
>
>
> - if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] <
> 2)
>
> + if (last < 2 && curByte != matchByte)
>
> {
>
> p->backRes = MARK_LIT;
>
> return 1;
>
> @@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> LitEnc_Matched_GetPrice(probs, curByte, matchByte,
> p->ProbPrices) :
>
> LitEnc_GetPrice(probs, curByte, p->ProbPrices));
>
> }
>
> -
>
> +
>
> MakeAs_Lit(&p->opt[1]);
>
>
>
> matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
>
>
>
> - if (matchByte == curByte)
>
> + // 18.06
>
> + if (matchByte == curByte && repLens[0] == 0)
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p,
> p->state, posState);
>
> if (shortRepPrice < p->opt[1].price)
>
> @@ -1160,16 +1226,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[1].price = shortRepPrice;
>
> MakeAs_ShortRep(&p->opt[1]);
>
> }
>
> + if (last < 2)
>
> + {
>
> + p->backRes = p->opt[1].dist;
>
> + return 1;
>
> + }
>
> }
>
> -
>
> - last = (mainLen >= repLens[repMaxIndex] ? mainLen :
> repLens[repMaxIndex]);
>
> -
>
> - if (last < 2)
>
> - {
>
> - p->backRes = p->opt[1].dist;
>
> - return 1;
>
> - }
>
> -
>
> +
>
> p->opt[1].len = 1;
>
>
>
> p->opt[0].reps[0] = reps[0];
>
> @@ -1177,13 +1240,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[0].reps[2] = reps[2];
>
> p->opt[0].reps[3] = reps[3];
>
>
>
> - {
>
> - unsigned len = last;
>
> - do
>
> - p->opt[len--].price = kInfinityPrice;
>
> - while (len >= 2);
>
> - }
>
> -
>
> // ---------- REP ----------
>
>
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> @@ -1195,13 +1251,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)repLen - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState,
> repLen);
>
> COptimal *opt = &p->opt[repLen];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = repLen;
>
> - opt->dist = i;
>
> + opt->len = (UInt32)repLen;
>
> + opt->dist = (UInt32)i;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1211,38 +1267,41 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> // ---------- MATCH ----------
>
> {
>
> - unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
>
> + unsigned len = repLens[0] + 1;
>
> if (len <= mainLen)
>
> {
>
> unsigned offs = 0;
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[p->state]);
>
>
>
> - while (len > matches[offs])
>
> - offs += 2;
>
> + if (len < 2)
>
> + len = 2;
>
> + else
>
> + while (len > matches[offs])
>
> + offs += 2;
>
>
>
> for (; ; len++)
>
> {
>
> COptimal *opt;
>
> UInt32 dist = matches[(size_t)offs + 1];
>
> - UInt32 price2 = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice +
> GET_PRICE_LEN(&p->lenEnc, posState, len);
>
> unsigned lenToPosState = GetLenToPosState(len);
>
>
>
> if (dist < kNumFullDistances)
>
> - price2 += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> else
>
> {
>
> unsigned slot;
>
> GetPosSlot2(dist, slot);
>
> - price2 += p->alignPrices[dist & kAlignMask];
>
> - price2 += p->posSlotPrices[lenToPosState][slot];
>
> + price += p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenToPosState][slot];
>
> }
>
>
>
> opt = &p->opt[len];
>
>
>
> - if (price2 < opt->price)
>
> + if (price < opt->price)
>
> {
>
> - opt->price = price2;
>
> - opt->len = len;
>
> + opt->price = price;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> @@ -1277,16 +1336,43 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (;;)
>
> {
>
> - UInt32 numAvail, numAvailFull;
>
> + unsigned numAvail;
>
> + UInt32 numAvailFull;
>
> unsigned newLen, numPairs, prev, state, posState, startLen;
>
> - UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
>
> - Bool nextIsLit;
>
> + UInt32 litPrice, matchPrice, repMatchPrice;
>
> + BoolInt nextIsLit;
>
> Byte curByte, matchByte;
>
> const Byte *data;
>
> COptimal *curOpt, *nextOpt;
>
>
>
> if (++cur == last)
>
> - return Backward(p, cur);
>
> + break;
>
> +
>
> + // 18.06
>
> + if (cur >= kNumOpts - 64)
>
> + {
>
> + unsigned j, best;
>
> + UInt32 price = p->opt[cur].price;
>
> + best = cur;
>
> + for (j = cur + 1; j <= last; j++)
>
> + {
>
> + UInt32 price2 = p->opt[j].price;
>
> + if (price >= price2)
>
> + {
>
> + price = price2;
>
> + best = j;
>
> + }
>
> + }
>
> + {
>
> + unsigned delta = best - cur;
>
> + if (delta != 0)
>
> + {
>
> + MOVE_POS(p, delta);
>
> + }
>
> + }
>
> + cur = best;
>
> + break;
>
> + }
>
>
>
> newLen = ReadMatchDistances(p, &numPairs);
>
>
>
> @@ -1294,15 +1380,24 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> p->numPairs = numPairs;
>
> p->longestMatchLen = newLen;
>
> - return Backward(p, cur);
>
> + break;
>
> }
>
>
>
> curOpt = &p->opt[cur];
>
> +
>
> + position++;
>
> +
>
> + // we need that check here, if skip_items in p->opt are possible
>
> + /*
>
> + if (curOpt->price >= kInfinityPrice)
>
> + continue;
>
> + */
>
> +
>
> prev = cur - curOpt->len;
>
> -
>
> +
>
> if (curOpt->len == 1)
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (IsShortRep(curOpt))
>
> state = kShortRepNextStates[state];
>
> else
>
> @@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> if (curOpt->extra)
>
> {
>
> - prev -= curOpt->extra;
>
> + prev -= (unsigned)curOpt->extra;
>
> state = kState_RepAfterLit;
>
> if (curOpt->extra == 1)
>
> - state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit :
> kState_MatchAfterLit;
>
> + state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit :
> kState_MatchAfterLit);
>
> }
>
> else
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (dist < LZMA_NUM_REPS)
>
> state = kRepNextStates[state];
>
> else
>
> @@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
>
>
> - position++;
>
> posState = (position & p->pbMask);
>
>
>
> /*
>
> @@ -1391,13 +1485,25 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> < MATCH [ : LIT : REP_0 ]
>
> */
>
>
>
> - curPrice = curOpt->price;
>
> - litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
>
> + {
>
> + UInt32 curPrice = curOpt->price;
>
> + unsigned prob = p->isMatch[state][posState];
>
> + matchPrice = curPrice + GET_PRICE_1(prob);
>
> + litPrice = curPrice + GET_PRICE_0(prob);
>
> + }
>
>
>
> nextOpt = &p->opt[(size_t)cur + 1];
>
> nextIsLit = False;
>
>
>
> - // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
>
> + // here we can allow skip_items in p->opt, if we don't check
> (nextOpt->price < kInfinityPrice)
>
> + // 18.new.06
>
> + if ((nextOpt->price < kInfinityPrice
>
> + // && !IsLitState(state)
>
> + && matchByte == curByte)
>
> + || litPrice > nextOpt->price
>
> + )
>
> + litPrice = 0;
>
> + else
>
> {
>
> const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
>
> litPrice += (!IsLitState(state) ?
>
> @@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
>
>
>
> + numAvailFull = p->numAvail;
>
> + {
>
> + unsigned temp = kNumOpts - 1 - cur;
>
> + if (numAvailFull > temp)
>
> + numAvailFull = (UInt32)temp;
>
> + }
>
> +
>
> + // 18.06
>
> // ---------- SHORT_REP ----------
>
> - // if (IsLitState(state)) // 18.new
>
> + if (IsLitState(state)) // 18.new
>
> if (matchByte == curByte)
>
> - // if (repMatchPrice < nextOpt->price) // 18.new
>
> - if (nextOpt->len < 2
>
> + if (repMatchPrice < nextOpt->price) // 18.new
>
> + // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
>
> + if (
>
> + // nextOpt->price >= kInfinityPrice ||
>
> + nextOpt->len < 2 // we can check nextOpt->len, if skip items
> are not allowed in p->opt
>
> || (nextOpt->dist != 0
>
> - && nextOpt->extra <= 1 // 17.old
>
> - ))
>
> + // && nextOpt->extra <= 1 // 17.old
>
> + )
>
> + )
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state,
> posState);
>
> - if (shortRepPrice <= nextOpt->price) // 17.old
>
> - // if (shortRepPrice < nextOpt->price) // 18.new
>
> + // if (shortRepPrice <= nextOpt->price) // 17.old
>
> + if (shortRepPrice < nextOpt->price) // 18.new
>
> {
>
> nextOpt->price = shortRepPrice;
>
> nextOpt->len = 1;
>
> @@ -1436,13 +1553,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - numAvailFull = p->numAvail;
>
> - {
>
> - UInt32 temp = kNumOpts - 1 - cur;
>
> - if (numAvailFull > temp)
>
> - numAvailFull = temp;
>
> - }
>
> -
>
> if (numAvailFull < 2)
>
> continue;
>
> numAvail = (numAvailFull <= p->numFastBytes ? numAvailFull :
> p->numFastBytes);
>
> @@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
>
>
> // ---------- LIT : REP_0 ----------
>
>
>
> - if (
>
> - // litPrice != 0 && // 18.new
>
> - !nextIsLit
>
> + if (!nextIsLit
>
> + && litPrice != 0 // 18.new
>
> && matchByte != curByte
>
> && numAvailFull > 2)
>
> {
>
> @@ -1465,8 +1574,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> for (len = 3; len < limit && data[len] == data2[len]; len++)
>
> - {
>
> - }
>
> + {}
>
>
>
> {
>
> unsigned state2 = kLiteralNextStates[state];
>
> @@ -1474,8 +1582,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
>
> {
>
> unsigned offset = cur + len;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
>
>
> // do
>
> {
>
> @@ -1483,14 +1592,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> COptimal *opt;
>
> len--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = 0;
>
> opt->extra = 1;
>
> }
>
> @@ -1502,6 +1611,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> }
>
>
>
> startLen = 2; /* speed optimization */
>
> +
>
> {
>
> // ---------- REP ----------
>
> unsigned repIndex = 0; // 17.old
>
> @@ -1514,24 +1624,28 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
>
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
>
>
> // if (len < startLen) continue; // 18.new: speed optimization
>
>
>
> - while (last < cur + len)
>
> - p->opt[++last].price = kInfinityPrice;
>
> + {
>
> + unsigned offset = cur + len;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
> {
>
> unsigned len2 = len;
>
> price = repMatchPrice + GetPrice_PureRep(p, repIndex, state,
> posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)len2 - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState, len2);
>
> COptimal *opt = &p->opt[cur + len2];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> - opt->dist = repIndex;
>
> + opt->len = (UInt32)len2;
>
> + opt->dist = (UInt32)repIndex;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1551,15 +1665,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> -
>
> - len2 -= len;
>
> - if (len2 >= 3)
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> {
>
> unsigned state2 = kRepNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> - price +=
>
> - p->repLenEnc.prices[posState][(size_t)len - 2]
>
> + price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
>
> + GET_PRICE_0(p->isMatch[state2][posState2])
>
> + LitEnc_Matched_GetPrice(LIT_PROBS(position + len,
> data[(size_t)len - 1]),
>
> data[len], data2[len], p->ProbPrices);
>
> @@ -1570,31 +1683,40 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
>
>
> price += GetPrice_Rep_0(p, state2, posState2);
>
> +
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
> +
>
> + len2 -= len;
>
> + // if (len2 >= 3)
>
> + {
>
> {
>
> unsigned offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> - unsigned price2;
>
> + UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> - opt->dist = repIndex;
>
> + opt->dist = (UInt32)repIndex;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> }
>
> + }
>
> }
>
> }
>
> }
>
> @@ -1606,17 +1728,23 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> newLen = numAvail;
>
> for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
>
> - matches[numPairs] = newLen;
>
> + matches[numPairs] = (UInt32)newLen;
>
> numPairs += 2;
>
> }
>
>
>
> + // startLen = 2; /* speed optimization */
>
> +
>
> if (newLen >= startLen)
>
> {
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[state]);
>
> UInt32 dist;
>
> unsigned offs, posSlot, len;
>
> - while (last < cur + newLen)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + {
>
> + unsigned offset = cur + newLen;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
>
>
> offs = 0;
>
> while (startLen > matches[offs])
>
> @@ -1628,27 +1756,29 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (len = /*2*/ startLen; ; len++)
>
> {
>
> - UInt32 price = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc,
> posState, len);
>
> {
>
> COptimal *opt;
>
> - unsigned lenToPosState = len - 2; lenToPosState =
> GetLenToPosState2(lenToPosState);
>
> + unsigned lenNorm = len - 2;
>
> + lenNorm = GetLenToPosState2(lenNorm);
>
> if (dist < kNumFullDistances)
>
> - price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenNorm][dist &
> (kNumFullDistances - 1)];
>
> else
>
> - price += p->posSlotPrices[lenToPosState][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenNorm][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
>
>
> opt = &p->opt[cur + len];
>
> if (price < opt->price)
>
> {
>
> opt->price = price;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> }
>
>
>
> - if (/*_maxMode && */ len == matches[offs])
>
> + if (len == matches[offs])
>
> {
>
> + // if (p->_maxMode) {
>
> // MATCH : LIT : REP_0
>
>
>
> const Byte *data2 = data - dist - 1;
>
> @@ -1657,11 +1787,17 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> + {
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
>
>
> len2 -= len;
>
>
>
> - if (len2 >= 3)
>
> + // if (len2 >= 3)
>
> {
>
> unsigned state2 = kMatchNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> @@ -1677,27 +1813,30 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price += GetPrice_Rep_0(p, state2, posState2);
>
>
>
> offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> +
>
> + }
>
>
>
> offs += 2;
>
> if (offs == numPairs)
>
> @@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
> }
>
> +
>
> + do
>
> + p->opt[last].price = kInfinityPrice;
>
> + while (--last);
>
> +
>
> + return Backward(p, cur);
>
> }
>
>
>
>
>
> @@ -1735,6 +1880,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> p->backRes = MARK_LIT;
>
> if (numAvail < 2)
>
> return 1;
>
> + // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
>
> if (numAvail > LZMA_MATCH_LEN_MAX)
>
> numAvail = LZMA_MATCH_LEN_MAX;
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> @@ -1746,10 +1892,11 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> const Byte *data2 = data - p->reps[i];
>
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> if (len >= p->numFastBytes)
>
> {
>
> - p->backRes = i;
>
> + p->backRes = (UInt32)i;
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> }
>
> @@ -1793,7 +1940,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> || (repLen + 2 >= mainLen && mainDist >= (1 << 9))
>
> || (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
>
> {
>
> - p->backRes = repIndex;
>
> + p->backRes = (UInt32)repIndex;
>
> MOVE_POS(p, repLen - 1)
>
> return repLen;
>
> }
>
> @@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p,
> UInt32 nowPos)
> }
>
>
>
>
>
> -
>
> -static void FillAlignPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
>
> {
>
> unsigned i;
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> const CLzmaProb *probs = p->posAlignEncoder;
>
> - p->alignPriceCount = 0;
>
> + // p->alignPriceCount = 0;
>
> for (i = 0; i < kAlignTableSize / 2; i++)
>
> {
>
> UInt32 price = 0;
>
> - unsigned symbol = i;
>
> + unsigned sym = i;
>
> unsigned m = 1;
>
> unsigned bit;
>
> UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> prob = probs[m];
>
> p->alignPrices[i ] = price + GET_PRICEa_0(prob);
>
> p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
>
> @@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
> }
>
>
>
>
>
> -static void FillDistancesPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
>
> {
>
> + // int y; for (y = 0; y < 100; y++) {
>
> +
>
> UInt32 tempPrices[kNumFullDistances];
>
> - unsigned i, lenToPosState;
>
> + unsigned i, lps;
>
>
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> p->matchPriceCount = 0;
>
>
>
> - for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
>
> + for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
>
> {
>
> unsigned posSlot = GetPosSlot1(i);
>
> - unsigned footerBits = ((posSlot >> 1) - 1);
>
> + unsigned footerBits = (posSlot >> 1) - 1;
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> + const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
>
> // tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base,
> footerBits, i - base, p->ProbPrices);
>
> -
>
> - const CLzmaProb *probs = p->posEncoders + base;
>
> UInt32 price = 0;
>
> unsigned m = 1;
>
> - unsigned symbol = i - base;
>
> + unsigned sym = i;
>
> + unsigned offset = (unsigned)1 << footerBits;
>
> + base += i;
>
> +
>
> + if (footerBits)
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> price += GET_PRICEa(probs[m], bit);
>
> m = (m << 1) + bit;
>
> }
>
> while (--footerBits);
>
> - tempPrices[i] = price;
>
> +
>
> + {
>
> + unsigned prob = probs[m];
>
> + tempPrices[base ] = price + GET_PRICEa_0(prob);
>
> + tempPrices[base + offset] = price + GET_PRICEa_1(prob);
>
> + }
>
> }
>
>
>
> - for (lenToPosState = 0; lenToPosState < kNumLenToPosStates;
> lenToPosState++)
>
> + for (lps = 0; lps < kNumLenToPosStates; lps++)
>
> {
>
> - unsigned posSlot;
>
> - const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
>
> - UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
>
> - unsigned distTableSize = p->distTableSize;
>
> - const CLzmaProb *probs = encoder;
>
> - for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
>
> + unsigned slot;
>
> + unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
>
> + UInt32 *posSlotPrices = p->posSlotPrices[lps];
>
> + const CLzmaProb *probs = p->posSlotEncoder[lps];
>
> +
>
> + for (slot = 0; slot < distTableSize2; slot++)
>
> {
>
> - // posSlotPrices[posSlot] = RcTree_GetPrice(encoder,
> kNumPosSlotBits, posSlot, p->ProbPrices);
>
> - UInt32 price = 0;
>
> + // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits,
> slot, p->ProbPrices);
>
> + UInt32 price;
>
> unsigned bit;
>
> - unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
>
> - UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
>
> - posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
>
> - posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
>
> + unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
>
> + unsigned prob;
>
> + bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
>
> + posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
>
> + posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
>
> }
>
> - for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
>
> - posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> -
>
> +
>
> {
>
> - UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
>
> + UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> + for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
>
> {
>
> - distancesPrices[0] = posSlotPrices[0];
>
> - distancesPrices[1] = posSlotPrices[1];
>
> - distancesPrices[2] = posSlotPrices[2];
>
> - distancesPrices[3] = posSlotPrices[3];
>
> + posSlotPrices[(size_t)slot * 2 ] += delta;
>
> + posSlotPrices[(size_t)slot * 2 + 1] += delta;
>
> + delta += ((UInt32)1 << kNumBitPriceShiftBits);
>
> }
>
> + }
>
> +
>
> + {
>
> + UInt32 *dp = p->distancesPrices[lps];
>
> +
>
> + dp[0] = posSlotPrices[0];
>
> + dp[1] = posSlotPrices[1];
>
> + dp[2] = posSlotPrices[2];
>
> + dp[3] = posSlotPrices[3];
>
> +
>
> for (i = 4; i < kNumFullDistances; i += 2)
>
> {
>
> UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
>
> - distancesPrices[i ] = slotPrice + tempPrices[i];
>
> - distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
>
> + dp[i ] = slotPrice + tempPrices[i];
>
> + dp[i + 1] = slotPrice + tempPrices[i + 1];
>
> }
>
> }
>
> }
>
> + // }
>
> }
>
>
>
>
>
> @@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> if (len != 1)
>
> {
>
> LenEnc_Encode(&p->repLenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->repLenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->repLenEnc, posState,
> &p->repLenProbs, p->ProbPrices);
>
> -
>
> + --p->repLenEncCounter;
>
> p->state = kRepNextStates[p->state];
>
> }
>
> }
>
> @@ -2260,9 +2422,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> p->state = kMatchNextStates[p->state];
>
>
>
> LenEnc_Encode(&p->lenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->lenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->lenEnc, posState,
> &p->lenProbs, p->ProbPrices);
>
> + // --p->lenEnc.counter;
>
>
>
> dist -= LZMA_NUM_REPS;
>
> p->reps[3] = p->reps[2];
>
> @@ -2274,17 +2434,17 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> GetPosSlot(dist, posSlot);
>
> // RcTree_Encode_PosSlot(&p->rc,
> p->posSlotEncoder[GetLenToPosState(len)], posSlot);
>
> {
>
> - UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
>
> + UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
>
> range = p->rc.range;
>
> probs = p->posSlotEncoder[GetLenToPosState(len)];
>
> do
>
> {
>
> - CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
>
> - UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
>
> - symbol <<= 1;
>
> + CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
>
> + UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
>
> + sym <<= 1;
>
> RC_BIT(&p->rc, prob, bit);
>
> }
>
> - while (symbol < (1 << kNumPosSlotBits * 2));
>
> + while (sym < (1 << kNumPosSlotBits * 2));
>
> p->rc.range = range;
>
> }
>
>
>
> @@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> if (dist < kNumFullDistances)
>
> {
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> - RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, dist - base);
>
> + RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, (unsigned)(dist /* - base */));
>
> }
>
> else
>
> {
>
> @@ -2331,14 +2491,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> bit = dist & 1; dist >>= 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit); m = (m << 1) + bit;
>
> bit = dist & 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit);
>
> p->rc.range = range;
>
> - p->alignPriceCount++;
>
> + // p->alignPriceCount++;
>
> }
>
> }
>
> }
>
> }
>
> }
>
>
>
> - nowPos32 += len;
>
> + nowPos32 += (UInt32)len;
>
> p->additionalOffset -= len;
>
>
>
> if (p->additionalOffset == 0)
>
> @@ -2347,10 +2507,27 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
>
>
> if (!p->fastMode)
>
> {
>
> - if (p->matchPriceCount >= (1 << 7))
>
> + /*
>
> + if (p->alignPriceCount >= 16) // kAlignTableSize
>
> + FillAlignPrices(p);
>
> + if (p->matchPriceCount >= 128)
>
> FillDistancesPrices(p);
>
> - if (p->alignPriceCount >= kAlignTableSize)
>
> + if (p->lenEnc.counter <= 0)
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + */
>
> + if (p->matchPriceCount >= 64)
>
> + {
>
> FillAlignPrices(p);
>
> + // { int y; for (y = 0; y < 100; y++) {
>
> + FillDistancesPrices(p);
>
> + // }}
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + }
>
> + if (p->repLenEncCounter <= 0)
>
> + {
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> + LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb,
> &p->repLenProbs, p->ProbPrices);
>
> + }
>
> }
>
>
>
> if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
>
> @@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
>
>
> p->optEnd = 0;
>
> p->optCur = 0;
>
> +
>
> + {
>
> + for (i = 0; i < kNumOpts; i++)
>
> + p->opt[i].price = kInfinityPrice;
>
> + }
>
> +
>
> p->additionalOffset = 0;
>
>
>
> p->pbMask = (1 << p->pb) - 1;
>
> p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
>
> }
>
>
>
> +
>
> void LzmaEnc_InitPrices(CLzmaEnc *p)
>
> {
>
> if (!p->fastMode)
>
> @@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
> p->lenEnc.tableSize =
>
> p->repLenEnc.tableSize =
>
> p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
>
> +
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> +
>
> LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs,
> p->ProbPrices);
>
> LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs,
> p->ProbPrices);
>
> }
>
> @@ -2583,7 +2770,7 @@ typedef struct
> ISeqOutStream vt;
>
> Byte *data;
>
> SizeT rem;
>
> - Bool overflow;
>
> + BoolInt overflow;
>
> } CLzmaEnc_SeqOutStreamBuf;
>
>
>
> static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void
> *data, size_t size)
>
> @@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle
> pp)
> }
>
>
>
>
>
> -SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
>
> +SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
>
> Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32
> *unpackSize)
>
> {
>
> CLzmaEnc *p = (CLzmaEnc *)pp;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> index d7426d38a4..04f9d98e7d 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> @@ -1,6 +1,28 @@
> HISTORY of the LZMA SDK
>
> -----------------------
>
>
>
> +19.00 2019-02-21
>
> +-------------------------
>
> +- Encryption strength for 7z archives was increased:
>
> + the size of random initialization vector was increased from 64-bit to
> 128-bit,
>
> + and the pseudo-random number generator was improved.
>
> +- The bug in 7zIn.c code was fixed.
>
> +
>
> +
>
> +18.06 2018-12-30
>
> +-------------------------
>
> +- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
>
> + and there are minor changes in compression ratio.
>
> +- Some bugs were fixed.
>
> +- The bug in 7-Zip 18.02-18.05 was fixed:
>
> + There was memory leak in multithreading xz decoder -
> XzDecMt_Decode(),
>
> + if xz stream contains only one block.
>
> +- The changes for MSVS compiler makefiles:
>
> + - the makefiles now use "PLATFORM" macroname with values (x64, x86,
> arm64)
>
> + instead of "CPU" macroname with values (AMD64, ARM64).
>
> + - the makefiles by default now use static version of the run-time
library.
>
> +
>
> +
>
> 18.05 2018-04-30
>
> -------------------------
>
> - The speed for LZMA/LZMA2 compressing was increased
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> index a3deb20943..e17cd4fe1a 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> @@ -1,4 +1,4 @@
> -LZMA SDK 18.05
>
> +LZMA SDK 19.00
>
> --------------
>
>
>
> LZMA SDK provides the documentation, samples, header files,
>
> --
> 2.27.0.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
2020-12-17 1:30 ` 回复: " gaoliming
@ 2020-12-20 13:33 ` Liu, WeiX C
2020-12-21 1:10 ` 回复: " gaoliming
0 siblings, 1 reply; 7+ messages in thread
From: Liu, WeiX C @ 2020-12-20 13:33 UTC (permalink / raw)
To: gaoliming, devel@edk2.groups.io; +Cc: 'Laszlo Ersek'
[-- Attachment #1: Type: text/plain, Size: 110759 bytes --]
Hi liming,
I listed unittest result as below with Ovmf build, cause I have no apple machine so only list VS、GCC and clang tool chain .
VS2019
GCC5
CLANGPDB
IA32
Pass
Pass
Pass
X64
Pass
Pass
Pass
IA32x64
Pass
Pass
Pass
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn>
Sent: 2020年12月17日 9:30
To: Liu, WeiX C <weix.c.liu@intel.com>; devel@edk2.groups.io
Cc: 'Laszlo Ersek' <lersek@redhat.com>
Subject: 回复: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
Wei:
Can you list the unit test for this patch? Which tool chain and arch have
been verified?
Because LZMA is well used, I request to verify it on VS, GCC, XCODE and
CLANG tool chain for the different ARCHs.
Thanks
Liming
> -----邮件原件-----
> 发件人: Wei Liu <weix.c.liu@intel.com>
> 发送时间: 2020年12月15日 10:07
> 收件人: devel@edk2.groups.io
> 抄送: Wei Liu <weix.c.liu@intel.com>; Liming Gao
> <gaoliming@byosoft.com.cn>; Laszlo Ersek <lersek@redhat.com>
> 主题: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00
> version
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
> New formal release in https://www.7-zip.org/sdk.html is 19.00.
>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Signed-off-by: Wei Liu <weix.c.liu@intel.com>
> ---
> BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4
> ++--
> BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5
> +++--
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9
> +++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 164
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++++++++++++++++++++++++++++++++++++++--------------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14
> +++++++-------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32
> ++++++++++++++++----------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 733
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
++++++++++++++++++++++++++--------------------------------------------------
----
>
----------------------------------------------------------------------------
-----------------
>
----------------------------------------------------------------------------
-----------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 22
> ++++++++++++++++++++++
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
> 12 files changed, 710 insertions(+), 408 deletions(-)
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> index 7cf967a774..b2338e2598 100644
> --- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> +++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> @@ -1,3 +1,3 @@
> -LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
>
> -was placed in the public domain on 2018-04-30. It was
>
> +LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
>
> +was placed in the public domain on 2019-02-21. It was
>
> released on the http://www.7-zip.org/sdk.html website.
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> index bebdb9aa84..bf4706ee23 100644
> --- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> +++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> @@ -1,9 +1,9 @@
> /** @file
>
> LZMA Compress/Decompress tool (LzmaCompress)
>
>
>
> - Based on LZMA SDK 18.05:
>
> + Based on LZMA SDK 19.00:
>
> LzmaUtil.c -- Test application for LZMA compression
>
> - 2018-04-30 : Igor Pavlov : Public domain
>
> + 2019-02-21 : Igor Pavlov : Public domain
>
>
>
> Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
>
> @@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate
> memory";
> const char *kDataErrorMessage = "Data error";
>
> const char *kInvalidParamValMessage = "Invalid parameter value";
>
>
>
> -static Bool mQuietMode = False;
>
> +static BoolInt mQuietMode = False;
>
> static CONVERTER_TYPE mConType = NoConverter;
>
>
>
> UINT64 mDictionarySize = 28;
>
> @@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
> CFileOutStream outStream;
>
> int res;
>
> int encodeMode = 0;
>
> - Bool modeWasSet = False;
>
> + BoolInt modeWasSet = False;
>
> const char *inputFile = NULL;
>
> const char *outputFile = "file.tmp";
>
> int param;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> index 4977cdaa66..593f5aa259 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> @@ -1,5 +1,5 @@
> /* 7zTypes.h -- Basic types
>
> -2017-07-17 : Igor Pavlov : Public domain */
>
> +2018-08-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __7Z_TYPES_H
>
> #define __7Z_TYPES_H
>
> @@ -103,7 +103,8 @@ typedef UInt32 SizeT;
> typedef size_t SizeT;
>
> #endif
>
>
>
> -typedef int Bool;
>
> +typedef int BoolInt;
>
> +/* typedef BoolInt Bool; */
>
> #define True 1
>
> #define False 0
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> index ed3aa94270..0074c64be9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> @@ -1,7 +1,7 @@
> -#define MY_VER_MAJOR 18
>
> -#define MY_VER_MINOR 05
>
> +#define MY_VER_MAJOR 19
>
> +#define MY_VER_MINOR 00
>
> #define MY_VER_BUILD 0
>
> -#define MY_VERSION_NUMBERS "18.05"
>
> +#define MY_VERSION_NUMBERS "19.00"
>
> #define MY_VERSION MY_VERSION_NUMBERS
>
>
>
> #ifdef MY_CPU_NAME
>
> @@ -10,7 +10,7 @@
> #define MY_VERSION_CPU MY_VERSION
>
> #endif
>
>
>
> -#define MY_DATE "2018-04-30"
>
> +#define MY_DATE "2019-02-21"
>
> #undef MY_COPYRIGHT
>
> #undef MY_VERSION_COPYRIGHT_DATE
>
> #define MY_AUTHOR_NAME "Igor Pavlov"
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> index 7fb27282c7..5f74c1c0cb 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> @@ -1,5 +1,5 @@
> /* CpuArch.h -- CPU specific code
>
> -2017-09-04 : Igor Pavlov : Public domain */
>
> +2018-02-18 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __CPU_ARCH_H
>
> #define __CPU_ARCH_H
>
> @@ -318,15 +318,16 @@ enum
>
>
> void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32
*d);
>
>
>
> -Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> +BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> int x86cpuid_GetFirm(const Cx86cpuid *p);
>
>
>
> #define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) &
0xF))
>
> #define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) &
> 0xF))
>
> #define x86cpuid_GetStepping(ver) (ver & 0xF)
>
>
>
> -Bool CPU_Is_InOrder();
>
> -Bool CPU_Is_Aes_Supported();
>
> +BoolInt CPU_Is_InOrder();
>
> +BoolInt CPU_Is_Aes_Supported();
>
> +BoolInt CPU_IsSupported_PageGB();
>
>
>
> #endif
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> index 6ea82a9b53..4eefc17dd2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> @@ -1,5 +1,5 @@
> /* LzFind.c -- Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-07-08 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -138,7 +138,7 @@ static void
> MatchFinder_SetDefaultSettings(CMatchFinder *p)
>
>
> void MatchFinder_Construct(CMatchFinder *p)
>
> {
>
> - UInt32 i;
>
> + unsigned i;
>
> p->bufferBase = NULL;
>
> p->directInput = 0;
>
> p->hash = NULL;
>
> @@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
>
>
> for (i = 0; i < 256; i++)
>
> {
>
> - UInt32 r = i;
>
> + UInt32 r = (UInt32)i;
>
> unsigned j;
>
> for (j = 0; j < 8; j++)
>
> r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
>
> @@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
> MatchFinder_ReduceOffsets(p, subValue);
>
> }
>
>
>
> +
>
> +MY_NO_INLINE
>
> static void MatchFinder_CheckLimits(CMatchFinder *p)
>
> {
>
> if (p->pos == kMaxValForNormalize)
>
> @@ -379,10 +381,16 @@ static void
> MatchFinder_CheckLimits(CMatchFinder *p)
> MatchFinder_SetLimits(p);
>
> }
>
>
>
> -static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> +
>
> +/*
>
> + (lenLimit > maxLen)
>
> +*/
>
> +MY_FORCE_INLINE
>
> +static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> - UInt32 *distances, UInt32 maxLen)
>
> + UInt32 *distances, unsigned maxLen)
>
> {
>
> + /*
>
> son[_cyclicBufferPos] = curMatch;
>
> for (;;)
>
> {
>
> @@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos,
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> return distances;
>
> @@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32
> lenLimit, UInt32 curMatch, UInt32 pos,
> }
>
> }
>
> }
>
> + */
>
> +
>
> + const Byte *lim = cur + lenLimit;
>
> + son[_cyclicBufferPos] = curMatch;
>
> + do
>
> + {
>
> + UInt32 delta = pos - curMatch;
>
> + if (delta >= _cyclicBufferSize)
>
> + break;
>
> + {
>
> + ptrdiff_t diff;
>
> + curMatch = son[_cyclicBufferPos - delta + ((delta >
_cyclicBufferPos) ?
> _cyclicBufferSize : 0)];
>
> + diff = (ptrdiff_t)0 - delta;
>
> + if (cur[maxLen] == cur[maxLen + diff])
>
> + {
>
> + const Byte *c = cur;
>
> + while (*c == c[diff])
>
> + {
>
> + if (++c == lim)
>
> + {
>
> + distances[0] = (UInt32)(lim - cur);
>
> + distances[1] = delta - 1;
>
> + return distances + 2;
>
> + }
>
> + }
>
> + {
>
> + unsigned len = (unsigned)(c - cur);
>
> + if (maxLen < len)
>
> + {
>
> + maxLen = len;
>
> + distances[0] = (UInt32)len;
>
> + distances[1] = delta - 1;
>
> + distances += 2;
>
> + }
>
> + }
>
> + }
>
> + }
>
> + }
>
> + while (--cutValue);
>
> +
>
> + return distances;
>
> }
>
>
>
> +
>
> +MY_FORCE_INLINE
>
> UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> UInt32 *distances, UInt32 maxLen)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> return distances;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = pair[0];
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const Byt
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = (UInt32)len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> + *ptr1 = pair0;
>
> *ptr0 = pair[1];
>
> return distances;
>
> }
>
> @@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const
> return;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> if (pb[len] == cur[len])
>
> {
>
> while (++len != lenLimit)
>
> @@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const
> p->buffer++; \
>
> if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
>
>
>
> -#define MOVE_POS_RET MOVE_POS return offset;
>
> +#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
>
>
>
> static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
>
>
>
> #define GET_MATCHES_HEADER2(minLen, ret_op) \
>
> - UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> - lenLimit = p->lenLimit; { if (lenLimit < minLen) {
MatchFinder_MovePos(p);
> ret_op; }} \
>
> + unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> + lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen)
> { MatchFinder_MovePos(p); ret_op; }} \
>
> cur = p->buffer;
>
>
>
> #define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen,
> return 0)
>
> @@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder
> *p) { MOVE_POS; }
> #define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos,
> p->cyclicBufferSize, p->cutValue
>
>
>
> #define GET_MATCHES_FOOTER(offset, maxLen) \
>
> - offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
>
> - distances + offset, maxLen) - distances); MOVE_POS_RET;
>
> + offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch,
> MF_PARAMS(p), \
>
> + distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
>
>
>
> #define SKIP_FOOTER \
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
> MOVE_POS;
>
>
>
> #define UPDATE_maxLen { \
>
> ptrdiff_t diff = (ptrdiff_t)0 - d2; \
>
> const Byte *c = cur + maxLen; \
>
> const Byte *lim = cur + lenLimit; \
>
> for (; c != lim; c++) if (*(c + diff) != *c) break; \
>
> - maxLen = (UInt32)(c - cur); }
>
> + maxLen = (unsigned)(c - cur); }
>
>
>
> static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(2)
>
> HASH2_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -561,7 +615,7 @@ static UInt32
> Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -572,7 +626,8 @@ UInt32
> Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, d2, maxLen, offset, pos;
>
> + UInt32 h2, d2, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(3)
>
>
>
> @@ -594,12 +649,12 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> UPDATE_maxLen
>
> - distances[0] = maxLen;
>
> + distances[0] = (UInt32)maxLen;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -609,7 +664,8 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -618,12 +674,12 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
>
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -632,7 +688,8 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -648,10 +705,10 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -674,13 +731,13 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -741,7 +798,8 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -750,12 +808,11 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> -
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -764,7 +821,8 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -780,7 +838,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -791,7 +849,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (maxLen < 3)
>
> maxLen = 3;
>
>
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances + offset, maxLen) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -808,13 +866,13 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -877,12 +935,12 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> p->hash[hv] = p->pos;
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances, 2) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> SKIP_FOOTER
>
> @@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> @@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> index 2563824fcd..df32146f92 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> @@ -1,5 +1,5 @@
> /* LzFindMt.c -- multithreaded Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-12-29 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -232,38 +232,57 @@ static void
> MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
>
>
> #define kEmptyHashValue 0
>
>
>
> -/* #define MFMT_GM_INLINE */
>
> +#define MFMT_GM_INLINE
>
>
>
> #ifdef MFMT_GM_INLINE
>
>
>
> -#define NO_INLINE MY_FAST_CALL
>
> +/*
>
> + we use size_t for _cyclicBufferPos instead of UInt32
>
> + to eliminate "movsx" BUG in old MSVC x64 compiler.
>
> +*/
>
>
>
> -static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> - UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> - UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit,
> UInt32 size, UInt32 *posRes)
>
> +MY_NO_INLINE
>
> +static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte
> *cur, CLzRef *son,
>
> + size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> + UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32
> *limit, UInt32 size, UInt32 *posRes)
>
> {
>
> do
>
> {
>
> - UInt32 *distances = _distances + 1;
>
> - UInt32 curMatch = pos - *hash++;
>
> + UInt32 *_distances = ++distances;
>
> + UInt32 delta = *hash++;
>
>
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> UInt32 cutValue = _cutValue;
>
> - UInt32 maxLen = _maxLen;
>
> - for (;;)
>
> + unsigned maxLen = (unsigned)_maxLen;
>
> +
>
> + /*
>
> + if (size > 1)
>
> {
>
> - UInt32 delta = pos - curMatch;
>
> - if (cutValue-- == 0 || delta >= _cyclicBufferSize)
>
> + UInt32 delta = *hash;
>
> + if (delta < _cyclicBufferSize)
>
> {
>
> - *ptr0 = *ptr1 = kEmptyHashValue;
>
> - break;
>
> + UInt32 cyc1 = _cyclicBufferPos + 1;
>
> + CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ?
> _cyclicBufferSize : 0)) << 1);
>
> + Byte b = *(cur + 1 - delta);
>
> + _distances[0] = pair[0];
>
> + _distances[1] = b;
>
> }
>
> + }
>
> + */
>
> + if (cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + }
>
> + else
>
> + for(;;)
>
> + {
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta +
> ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = *pair;
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32
> lenLimit, UInt32 pos, const Byte *
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> - *ptr0 = pair[1];
>
> + UInt32 pair1 = pair[1];
>
> + *ptr1 = pair0;
>
> + *ptr0 = pair1;
>
> break;
>
> }
>
> }
>
> }
>
> - if (pb[len] < cur[len])
>
> - {
>
> - *ptr1 = curMatch;
>
> - ptr1 = pair + 1;
>
> - curMatch = *ptr1;
>
> - len1 = len;
>
> - }
>
> - else
>
> {
>
> - *ptr0 = curMatch;
>
> - ptr0 = pair;
>
> - curMatch = *ptr0;
>
> - len0 = len;
>
> + UInt32 curMatch = pos - delta;
>
> + // delta = pos - *pair;
>
> + // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >>
31];
>
> + if (pb[len] < cur[len])
>
> + {
>
> + delta = pos - pair[1];
>
> + *ptr1 = curMatch;
>
> + ptr1 = pair + 1;
>
> + len1 = len;
>
> + }
>
> + else
>
> + {
>
> + delta = pos - *pair;
>
> + *ptr0 = curMatch;
>
> + ptr0 = pair;
>
> + len0 = len;
>
> + }
>
> }
>
> }
>
> + if (--cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + break;
>
> + }
>
> }
>
> pos++;
>
> _cyclicBufferPos++;
>
> cur++;
>
> {
>
> UInt32 num = (UInt32)(distances - _distances);
>
> - *_distances = num - 1;
>
> - _distances += num;
>
> - limit -= num;
>
> + _distances[-1] = num;
>
> }
>
> }
>
> - while (limit > 0 && --size != 0);
>
> + while (distances < limit && --size != 0);
>
> *posRes = pos;
>
> - return limit;
>
> + return distances;
>
> }
>
>
>
> #endif
>
>
>
> +
>
> +
>
> static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
>
> {
>
> UInt32 numProcessed = 0;
>
> UInt32 curPos = 2;
>
> - UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
>
> + UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
>
>
>
> distances[1] = p->hashNumAvail;
>
>
>
> @@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p,
> UInt32 *distances)
> #else
>
> {
>
> UInt32 posRes;
>
> - curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer,
p->son,
> cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> - distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
>
> + curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer,
> p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> + distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos,
>
> + distances + limit,
>
> + size, &posRes) - distances);
>
> p->hashBufPos += posRes - pos;
>
> cyclicBufferPos += posRes - pos;
>
> p->buffer += posRes - pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> index 3d86c788f3..fdd17008c2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> @@ -1,5 +1,5 @@
> /* LzFindMt.h -- multithreaded Match finder for LZ algorithms
>
> -2017-04-03 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __LZ_FIND_MT_H
>
> #define __LZ_FIND_MT_H
>
> @@ -19,10 +19,10 @@ EXTERN_C_BEGIN
>
>
> typedef struct _CMtSync
>
> {
>
> - Bool wasCreated;
>
> - Bool needStart;
>
> - Bool exit;
>
> - Bool stopWriting;
>
> + BoolInt wasCreated;
>
> + BoolInt needStart;
>
> + BoolInt exit;
>
> + BoolInt stopWriting;
>
>
>
> CThread thread;
>
> CAutoResetEvent canStart;
>
> @@ -30,8 +30,8 @@ typedef struct _CMtSync
> CAutoResetEvent wasStopped;
>
> CSemaphore freeSemaphore;
>
> CSemaphore filledSemaphore;
>
> - Bool csWasInitialized;
>
> - Bool csWasEntered;
>
> + BoolInt csWasInitialized;
>
> + BoolInt csWasEntered;
>
> CCriticalSection cs;
>
> UInt32 numProcessedBlocks;
>
> } CMtSync;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> index 962b94bb63..4d1576419f 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> @@ -1,13 +1,13 @@
> /* LzmaDec.c -- LZMA Decoder
>
> -2018-02-28 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> +#include <string.h>
>
> +
>
> /* #include "CpuArch.h" */
>
> #include "LzmaDec.h"
>
>
>
> -#include <string.h>
>
> -
>
> #define kNumTopBits 24
>
> #define kTopValue ((UInt32)1 << kNumTopBits)
>
>
>
> @@ -19,7 +19,7 @@
>
>
> #define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code <<
8)
> | (*buf++); }
>
>
>
> -#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt +
> ((kBitModelTotal - ttt) >> kNumMoveBits));
>
> #define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt
> - (ttt >> kNumMoveBits));
>
> #define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
>
> @@ -66,7 +66,7 @@
>
>
> #define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit)
> return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
>
>
>
> -#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0_CHECK range = bound;
>
> #define UPDATE_1_CHECK range -= bound; code -= bound;
>
> #define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
>
> @@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> curLen = ((rem < len) ? (unsigned)rem : len);
>
> pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
>
>
>
> - processedPos += curLen;
>
> + processedPos += (UInt32)curLen;
>
>
>
> len -= curLen;
>
> if (curLen <= dicBufSize - pos)
>
> @@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> Byte *dest = dic + dicPos;
>
> ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
>
> const Byte *lim = dest + curLen;
>
> - dicPos += curLen;
>
> + dicPos += (SizeT)curLen;
>
> do
>
> *(dest) = (Byte)*(dest + src);
>
> while (++dest != lim);
>
> @@ -572,14 +572,14 @@ int MY_FAST_CALL
> LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
> p->buf = buf;
>
> p->range = range;
>
> p->code = code;
>
> - p->remainLen = len;
>
> + p->remainLen = (UInt32)len;
>
> p->dicPos = dicPos;
>
> p->processedPos = processedPos;
>
> p->reps[0] = rep0;
>
> p->reps[1] = rep1;
>
> p->reps[2] = rep2;
>
> p->reps[3] = rep3;
>
> - p->state = state;
>
> + p->state = (UInt32)state;
>
>
>
> return SZ_OK;
>
> }
>
> @@ -601,8 +601,8 @@ static void MY_FAST_CALL
> LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
> if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
>
> p->checkDicSize = p->prop.dicSize;
>
>
>
> - p->processedPos += len;
>
> - p->remainLen -= len;
>
> + p->processedPos += (UInt32)len;
>
> + p->remainLen -= (UInt32)len;
>
> while (len != 0)
>
> {
>
> len--;
>
> @@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const
> CLzmaDec *p, const Byte *buf, SizeT inS
> }
>
>
>
>
>
> -void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
>
> +void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt
> initState)
>
> {
>
> p->remainLen = kMatchSpecLenStart + 1;
>
> p->tempBufSize = 0;
>
> @@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> p->tempBufSize = rem;
>
> if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
>
> {
>
> - int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
>
> + int dummyRes = LzmaDec_TryDummy(p, p->tempBuf,
> (SizeT)rem);
>
> if (dummyRes == DUMMY_ERROR)
>
> {
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> *status = LZMA_STATUS_NEEDS_MORE_INPUT;
>
> return SZ_OK;
>
> }
>
> @@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> return SZ_ERROR_FAIL; /* some internal error */
>
> lookAhead -= rem;
>
> }
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> src += lookAhead;
>
> - inSize -= lookAhead;
>
> + inSize -= (SizeT)lookAhead;
>
> p->tempBufSize = 0;
>
> }
>
> }
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> index e281716fee..14086fc4f9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> @@ -1,5 +1,5 @@
> /* LzmaEnc.c -- LZMA Encoder
>
> -2018-04-29 : Igor Pavlov : Public domain */
>
> +2019-01-10: Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
> #define kNumBitPriceShiftBits 4
>
> #define kBitPrice (1 << kNumBitPriceShiftBits)
>
>
>
> +#define REP_LEN_COUNT 64
>
> +
>
> void LzmaEncProps_Init(CLzmaEncProps *p)
>
> {
>
> p->level = 5;
>
> @@ -183,8 +185,11 @@ typedef struct
> } COptimal;
>
>
>
>
>
> -#define kNumOpts (1 << 12)
>
> -#define kPackReserve (1 + kNumOpts * 2)
>
> +// 18.06
>
> +#define kNumOpts (1 << 11)
>
> +#define kPackReserve (kNumOpts * 8)
>
> +// #define kNumOpts (1 << 12)
>
> +// #define kPackReserve (1 + kNumOpts * 2)
>
>
>
> #define kNumLenToPosStates 4
>
> #define kNumPosSlotBits 6
>
> @@ -236,10 +241,18 @@ typedef struct
> typedef struct
>
> {
>
> unsigned tableSize;
>
> - unsigned counters[LZMA_NUM_PB_STATES_MAX];
>
> UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
>
> + // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols *
> 2];
>
> + // UInt32 prices2[kLenNumSymbolsTotal];
>
> } CLenPriceEnc;
>
>
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
>
> +
>
> +/*
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) -
2) &
> (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >>
9)))
>
> +*/
>
>
>
> typedef struct
>
> {
>
> @@ -308,16 +321,18 @@ typedef struct
> unsigned lc, lp, pb;
>
> unsigned lclp;
>
>
>
> - Bool fastMode;
>
> - Bool writeEndMark;
>
> - Bool finished;
>
> - Bool multiThread;
>
> - Bool needInit;
>
> + BoolInt fastMode;
>
> + BoolInt writeEndMark;
>
> + BoolInt finished;
>
> + BoolInt multiThread;
>
> + BoolInt needInit;
>
> + // BoolInt _maxMode;
>
>
>
> UInt64 nowPos64;
>
>
>
> unsigned matchPriceCount;
>
> - unsigned alignPriceCount;
>
> + // unsigned alignPriceCount;
>
> + int repLenEncCounter;
>
>
>
> unsigned distTableSize;
>
>
>
> @@ -325,7 +340,7 @@ typedef struct
> SRes result;
>
>
>
> #ifndef _7ZIP_ST
>
> - Bool mtMode;
>
> + BoolInt mtMode;
>
> // begin of CMatchFinderMt is used in LZ thread
>
> CMatchFinderMt matchFinderMt;
>
> // end of CMatchFinderMt is used in BT and HASH threads
>
> @@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const
> CLzmaEncProps *props2)
> p->lp = props.lp;
>
> p->pb = props.pb;
>
> p->fastMode = (props.algo == 0);
>
> + // p->_maxMode = True;
>
> p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
>
> {
>
> unsigned numHashBytes = 4;
>
> @@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #ifdef _LZMA_ENC_USE_BRANCH
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> RC_BIT_PRE(p, prob) \
>
> - if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> + if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> else { (p)->low += newBound; range -= newBound; ttt -= ttt >>
> kNumMoveBits; } \
>
> *(prob) = (CLzmaProb)ttt; \
>
> RC_NORM(p) \
>
> @@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #else
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> UInt32 mask; \
>
> RC_BIT_PRE(p, prob) \
>
> - mask = 0 - (UInt32)symbol; \
>
> + mask = 0 - (UInt32)bit; \
>
> range &= mask; \
>
> mask &= newBound; \
>
> range -= mask; \
>
> (p)->low += mask; \
>
> - mask = (UInt32)symbol - 1; \
>
> + mask = (UInt32)bit - 1; \
>
> range += newBound & mask; \
>
> mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
>
> mask += ((1 << kNumMoveBits) - 1); \
>
> @@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p,
> CLzmaProb *prob)
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
>
> +static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
>
> {
>
> UInt32 range = p->range;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
>
> - CLzmaProb *prob = probs + (symbol >> 8);
>
> - UInt32 bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> + // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
>
> + CLzmaProb *prob = probs + (sym >> 8);
>
> + UInt32 bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte)
>
> +static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs,
> UInt32 sym, UInt32 matchByte)
>
> {
>
> UInt32 range = p->range;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> CLzmaProb *prob;
>
> UInt32 bit;
>
> matchByte <<= 1;
>
> - // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol
>>
> 8)), (symbol >> 7) & 1);
>
> - prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
>
> - bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >>
> 8)), (sym >> 7) & 1);
>
> + prob = probs + (offs + (matchByte & offs) + (sym >> 8));
>
> + bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> @@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> }
>
>
>
>
>
> -#define GET_PRICE(prob, symbol) \
>
> - p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICE(prob, bit) \
>
> + p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> -#define GET_PRICEa(prob, symbol) \
>
> - ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICEa(prob, bit) \
>
> + ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> #define GET_PRICE_0(prob) p->ProbPrices[(prob) >>
> kNumMoveReducingBits]
>
> #define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1))
>>
> kNumMoveReducingBits]
>
> @@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> #define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >>
> kNumMoveReducingBits]
>
>
>
>
>
> -static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol,
const
> CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const
> CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> - price += GET_PRICEa(probs[symbol], bit);
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> }
>
> - while (symbol >= 2);
>
> + while (sym >= 2);
>
> return price;
>
> }
>
>
>
>
>
> -static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> sym, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> matchByte <<= 1;
>
> - price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)],
> (symbol >> 7) & 1);
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)],
> (sym >> 7) & 1);
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> return price;
>
> }
>
>
>
>
>
> -static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, UInt32 symbol)
>
> +static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, unsigned sym)
>
> {
>
> UInt32 range = rc->range;
>
> unsigned m = 1;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - unsigned bit = symbol & 1;
>
> + unsigned bit = sym & 1;
>
> // RangeEnc_EncodeBit(rc, probs + m, bit);
>
> - symbol >>= 1;
>
> + sym >>= 1;
>
> RC_BIT(rc, probs + m, bit);
>
> m = (m << 1) | bit;
>
> }
>
> @@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
> p->high[i] = kProbInitValue;
>
> }
>
>
>
> -static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol,
> unsigned posState)
>
> +static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym,
> unsigned posState)
>
> {
>
> UInt32 range, ttt, newBound;
>
> CLzmaProb *probs = p->low;
>
> range = rc->range;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols)
>
> + if (sym >= kLenNumLowSymbols)
>
> {
>
> RC_BIT_1(rc, probs);
>
> probs += kLenNumLowSymbols;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols * 2)
>
> + if (sym >= kLenNumLowSymbols * 2)
>
> {
>
> RC_BIT_1(rc, probs);
>
> rc->range = range;
>
> - // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol -
> kLenNumLowSymbols * 2);
>
> - LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
>
> + // RcTree_Encode(rc, p->high, kLenNumHighBits, sym -
> kLenNumLowSymbols * 2);
>
> + LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
>
> return;
>
> }
>
> - symbol -= kLenNumLowSymbols;
>
> + sym -= kLenNumLowSymbols;
>
> }
>
>
>
> - // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, symbol);
>
> + // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, sym);
>
> {
>
> unsigned m;
>
> unsigned bit;
>
> RC_BIT_0(rc, probs);
>
> probs += (posState << (1 + kLenNumLowBits));
>
> - bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) +
bit;
>
> - bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) +
bit;
>
> - bit = symbol & 1; RC_BIT(rc, probs + m, bit);
>
> + bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
>
> + bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
>
> + bit = sym & 1; RC_BIT(rc, probs + m, bit);
>
> rc->range = range;
>
> }
>
> }
>
> @@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs,
> UInt32 startPrice, UInt32 *price
> }
>
>
>
>
>
> -MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
>
> - CLenPriceEnc *p, unsigned posState,
>
> +MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
>
> + CLenPriceEnc *p,
>
> + unsigned numPosStates,
>
> const CLenEnc *enc,
>
> const CProbPrice *ProbPrices)
>
> {
>
> - // int y; for (y = 0; y < 100; y++) {
>
> - UInt32 a;
>
> - unsigned i, numSymbols;
>
> + UInt32 b;
>
> +
>
> + {
>
> + unsigned prob = enc->low[0];
>
> + UInt32 a, c;
>
> + unsigned posState;
>
> + b = GET_PRICEa_1(prob);
>
> + a = GET_PRICEa_0(prob);
>
> + c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (posState = 0; posState < numPosStates; posState++)
>
> + {
>
> + UInt32 *prices = p->prices[posState];
>
> + const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> + SetPrices_3(probs, a, prices, ProbPrices);
>
> + SetPrices_3(probs + kLenNumLowSymbols, c, prices +
> kLenNumLowSymbols, ProbPrices);
>
> + }
>
> + }
>
>
>
> - UInt32 *prices = p->prices[posState];
>
> + /*
>
> {
>
> - const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> - SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
>
> + unsigned i;
>
> + UInt32 b;
>
> + a = GET_PRICEa_0(enc->low[0]);
>
> + for (i = 0; i < kLenNumLowSymbols; i++)
>
> + p->prices2[i] = a;
>
> a = GET_PRICEa_1(enc->low[0]);
>
> - SetPrices_3(probs + kLenNumLowSymbols, a +
> GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices +
> kLenNumLowSymbols, ProbPrices);
>
> + b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
>
> + p->prices2[i] = b;
>
> a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> }
>
> - numSymbols = p->tableSize;
>
> - p->counters[posState] = numSymbols;
>
> - for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
>
> + */
>
> +
>
> + // p->counter = numSymbols;
>
> + // p->counter = 64;
>
> +
>
> {
>
> - prices[i] = a +
>
> - // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> - LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2,
> ProbPrices);
>
> - /*
>
> - unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
>
> - UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> - UInt32 prob = enc->high[(1 << 7) + sym];
>
> - prices[i ] = price + GET_PRICEa_0(prob);
>
> - prices[i + 1] = price + GET_PRICEa_1(prob);
>
> - */
>
> - }
>
> - // }
>
> -}
>
> + unsigned i = p->tableSize;
>
> +
>
> + if (i > kLenNumLowSymbols * 2)
>
> + {
>
> + const CLzmaProb *probs = enc->high;
>
> + UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
>
> + i -= kLenNumLowSymbols * 2 - 1;
>
> + i >>= 1;
>
> + b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> + do
>
> + {
>
> + /*
>
> + p->prices2[i] = a +
>
> + // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> + LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
>
> + */
>
> + // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> + unsigned sym = --i + (1 << (kLenNumHighBits - 1));
>
> + UInt32 price = b;
>
> + do
>
> + {
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> + }
>
> + while (sym >= 2);
>
>
>
> -static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned
> numPosStates,
>
> - const CLenEnc *enc,
>
> - const CProbPrice *ProbPrices)
>
> -{
>
> - unsigned posState;
>
> - for (posState = 0; posState < numPosStates; posState++)
>
> - LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
>
> -}
>
> + {
>
> + unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits -
1))];
>
> + prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
>
> + prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
>
> + }
>
> + }
>
> + while (i);
>
>
>
> + {
>
> + unsigned posState;
>
> + size_t num = (p->tableSize - kLenNumLowSymbols * 2) *
> sizeof(p->prices[0][0]);
>
> + for (posState = 1; posState < numPosStates; posState++)
>
> + memcpy(p->prices[posState] + kLenNumLowSymbols * 2,
> p->prices[0] + kLenNumLowSymbols * 2, num);
>
> + }
>
> + }
>
> + }
>
> +}
>
>
>
> /*
>
> #ifdef SHOW_STAT
>
> @@ -917,7 +976,7 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc
> *p, unsigned numPosStates,
>
>
> #define MOVE_POS(p, num) { \
>
> p->additionalOffset += (num); \
>
> - p->matchFinder.Skip(p->matchFinderObj, (num)); }
>
> + p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
>
>
>
>
>
> static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
>
> @@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> const Byte *p2 = p1 + len;
>
> ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
>
> const Byte *lim = p1 + numAvail;
>
> - for (; p2 != lim && *p2 == p2[dif]; p2++);
>
> + for (; p2 != lim && *p2 == p2[dif]; p2++)
>
> + {}
>
> return (unsigned)(p2 - p1);
>
> }
>
> }
>
> @@ -977,7 +1037,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> + GET_PRICE_1(p->isRep[state]) \
>
> + GET_PRICE_0(p->isRepG0[state])
>
>
>
> -
>
> +MY_FORCE_INLINE
>
> static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex,
> size_t state, size_t posState)
>
> {
>
> UInt32 price;
>
> @@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p,
> unsigned cur)
> for (;;)
>
> {
>
> UInt32 dist = p->opt[cur].dist;
>
> - UInt32 len = p->opt[cur].len;
>
> - UInt32 extra = p->opt[cur].extra;
>
> + unsigned len = (unsigned)p->opt[cur].len;
>
> + unsigned extra = (unsigned)p->opt[cur].extra;
>
> cur -= len;
>
>
>
> if (extra)
>
> {
>
> wr--;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> cur -= extra;
>
> len = extra;
>
> if (extra == 1)
>
> @@ -1045,7 +1105,7 @@ static unsigned Backward(CLzmaEnc *p, unsigned
> cur)
>
>
> wr--;
>
> p->opt[wr].dist = dist;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> }
>
> }
>
>
>
> @@ -1102,7 +1162,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> repLens[i] = 0;
>
> continue;
>
> }
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> repLens[i] = len;
>
> if (len > repLens[repMaxIndex])
>
> repMaxIndex = i;
>
> @@ -1111,7 +1172,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (repLens[repMaxIndex] >= p->numFastBytes)
>
> {
>
> unsigned len;
>
> - p->backRes = repMaxIndex;
>
> + p->backRes = (UInt32)repMaxIndex;
>
> len = repLens[repMaxIndex];
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> @@ -1128,8 +1189,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
> +
>
> + last = repLens[repMaxIndex];
>
> + if (last <= mainLen)
>
> + last = mainLen;
>
>
>
> - if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] <
> 2)
>
> + if (last < 2 && curByte != matchByte)
>
> {
>
> p->backRes = MARK_LIT;
>
> return 1;
>
> @@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> LitEnc_Matched_GetPrice(probs, curByte, matchByte,
> p->ProbPrices) :
>
> LitEnc_GetPrice(probs, curByte, p->ProbPrices));
>
> }
>
> -
>
> +
>
> MakeAs_Lit(&p->opt[1]);
>
>
>
> matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
>
>
>
> - if (matchByte == curByte)
>
> + // 18.06
>
> + if (matchByte == curByte && repLens[0] == 0)
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p,
> p->state, posState);
>
> if (shortRepPrice < p->opt[1].price)
>
> @@ -1160,16 +1226,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[1].price = shortRepPrice;
>
> MakeAs_ShortRep(&p->opt[1]);
>
> }
>
> + if (last < 2)
>
> + {
>
> + p->backRes = p->opt[1].dist;
>
> + return 1;
>
> + }
>
> }
>
> -
>
> - last = (mainLen >= repLens[repMaxIndex] ? mainLen :
> repLens[repMaxIndex]);
>
> -
>
> - if (last < 2)
>
> - {
>
> - p->backRes = p->opt[1].dist;
>
> - return 1;
>
> - }
>
> -
>
> +
>
> p->opt[1].len = 1;
>
>
>
> p->opt[0].reps[0] = reps[0];
>
> @@ -1177,13 +1240,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[0].reps[2] = reps[2];
>
> p->opt[0].reps[3] = reps[3];
>
>
>
> - {
>
> - unsigned len = last;
>
> - do
>
> - p->opt[len--].price = kInfinityPrice;
>
> - while (len >= 2);
>
> - }
>
> -
>
> // ---------- REP ----------
>
>
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> @@ -1195,13 +1251,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)repLen - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState,
> repLen);
>
> COptimal *opt = &p->opt[repLen];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = repLen;
>
> - opt->dist = i;
>
> + opt->len = (UInt32)repLen;
>
> + opt->dist = (UInt32)i;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1211,38 +1267,41 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> // ---------- MATCH ----------
>
> {
>
> - unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
>
> + unsigned len = repLens[0] + 1;
>
> if (len <= mainLen)
>
> {
>
> unsigned offs = 0;
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[p->state]);
>
>
>
> - while (len > matches[offs])
>
> - offs += 2;
>
> + if (len < 2)
>
> + len = 2;
>
> + else
>
> + while (len > matches[offs])
>
> + offs += 2;
>
>
>
> for (; ; len++)
>
> {
>
> COptimal *opt;
>
> UInt32 dist = matches[(size_t)offs + 1];
>
> - UInt32 price2 = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice +
> GET_PRICE_LEN(&p->lenEnc, posState, len);
>
> unsigned lenToPosState = GetLenToPosState(len);
>
>
>
> if (dist < kNumFullDistances)
>
> - price2 += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> else
>
> {
>
> unsigned slot;
>
> GetPosSlot2(dist, slot);
>
> - price2 += p->alignPrices[dist & kAlignMask];
>
> - price2 += p->posSlotPrices[lenToPosState][slot];
>
> + price += p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenToPosState][slot];
>
> }
>
>
>
> opt = &p->opt[len];
>
>
>
> - if (price2 < opt->price)
>
> + if (price < opt->price)
>
> {
>
> - opt->price = price2;
>
> - opt->len = len;
>
> + opt->price = price;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> @@ -1277,16 +1336,43 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (;;)
>
> {
>
> - UInt32 numAvail, numAvailFull;
>
> + unsigned numAvail;
>
> + UInt32 numAvailFull;
>
> unsigned newLen, numPairs, prev, state, posState, startLen;
>
> - UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
>
> - Bool nextIsLit;
>
> + UInt32 litPrice, matchPrice, repMatchPrice;
>
> + BoolInt nextIsLit;
>
> Byte curByte, matchByte;
>
> const Byte *data;
>
> COptimal *curOpt, *nextOpt;
>
>
>
> if (++cur == last)
>
> - return Backward(p, cur);
>
> + break;
>
> +
>
> + // 18.06
>
> + if (cur >= kNumOpts - 64)
>
> + {
>
> + unsigned j, best;
>
> + UInt32 price = p->opt[cur].price;
>
> + best = cur;
>
> + for (j = cur + 1; j <= last; j++)
>
> + {
>
> + UInt32 price2 = p->opt[j].price;
>
> + if (price >= price2)
>
> + {
>
> + price = price2;
>
> + best = j;
>
> + }
>
> + }
>
> + {
>
> + unsigned delta = best - cur;
>
> + if (delta != 0)
>
> + {
>
> + MOVE_POS(p, delta);
>
> + }
>
> + }
>
> + cur = best;
>
> + break;
>
> + }
>
>
>
> newLen = ReadMatchDistances(p, &numPairs);
>
>
>
> @@ -1294,15 +1380,24 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> p->numPairs = numPairs;
>
> p->longestMatchLen = newLen;
>
> - return Backward(p, cur);
>
> + break;
>
> }
>
>
>
> curOpt = &p->opt[cur];
>
> +
>
> + position++;
>
> +
>
> + // we need that check here, if skip_items in p->opt are possible
>
> + /*
>
> + if (curOpt->price >= kInfinityPrice)
>
> + continue;
>
> + */
>
> +
>
> prev = cur - curOpt->len;
>
> -
>
> +
>
> if (curOpt->len == 1)
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (IsShortRep(curOpt))
>
> state = kShortRepNextStates[state];
>
> else
>
> @@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> if (curOpt->extra)
>
> {
>
> - prev -= curOpt->extra;
>
> + prev -= (unsigned)curOpt->extra;
>
> state = kState_RepAfterLit;
>
> if (curOpt->extra == 1)
>
> - state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit :
> kState_MatchAfterLit;
>
> + state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit :
> kState_MatchAfterLit);
>
> }
>
> else
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (dist < LZMA_NUM_REPS)
>
> state = kRepNextStates[state];
>
> else
>
> @@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
>
>
> - position++;
>
> posState = (position & p->pbMask);
>
>
>
> /*
>
> @@ -1391,13 +1485,25 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> < MATCH [ : LIT : REP_0 ]
>
> */
>
>
>
> - curPrice = curOpt->price;
>
> - litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
>
> + {
>
> + UInt32 curPrice = curOpt->price;
>
> + unsigned prob = p->isMatch[state][posState];
>
> + matchPrice = curPrice + GET_PRICE_1(prob);
>
> + litPrice = curPrice + GET_PRICE_0(prob);
>
> + }
>
>
>
> nextOpt = &p->opt[(size_t)cur + 1];
>
> nextIsLit = False;
>
>
>
> - // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
>
> + // here we can allow skip_items in p->opt, if we don't check
> (nextOpt->price < kInfinityPrice)
>
> + // 18.new.06
>
> + if ((nextOpt->price < kInfinityPrice
>
> + // && !IsLitState(state)
>
> + && matchByte == curByte)
>
> + || litPrice > nextOpt->price
>
> + )
>
> + litPrice = 0;
>
> + else
>
> {
>
> const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
>
> litPrice += (!IsLitState(state) ?
>
> @@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
>
>
>
> + numAvailFull = p->numAvail;
>
> + {
>
> + unsigned temp = kNumOpts - 1 - cur;
>
> + if (numAvailFull > temp)
>
> + numAvailFull = (UInt32)temp;
>
> + }
>
> +
>
> + // 18.06
>
> // ---------- SHORT_REP ----------
>
> - // if (IsLitState(state)) // 18.new
>
> + if (IsLitState(state)) // 18.new
>
> if (matchByte == curByte)
>
> - // if (repMatchPrice < nextOpt->price) // 18.new
>
> - if (nextOpt->len < 2
>
> + if (repMatchPrice < nextOpt->price) // 18.new
>
> + // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
>
> + if (
>
> + // nextOpt->price >= kInfinityPrice ||
>
> + nextOpt->len < 2 // we can check nextOpt->len, if skip items
> are not allowed in p->opt
>
> || (nextOpt->dist != 0
>
> - && nextOpt->extra <= 1 // 17.old
>
> - ))
>
> + // && nextOpt->extra <= 1 // 17.old
>
> + )
>
> + )
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state,
> posState);
>
> - if (shortRepPrice <= nextOpt->price) // 17.old
>
> - // if (shortRepPrice < nextOpt->price) // 18.new
>
> + // if (shortRepPrice <= nextOpt->price) // 17.old
>
> + if (shortRepPrice < nextOpt->price) // 18.new
>
> {
>
> nextOpt->price = shortRepPrice;
>
> nextOpt->len = 1;
>
> @@ -1436,13 +1553,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - numAvailFull = p->numAvail;
>
> - {
>
> - UInt32 temp = kNumOpts - 1 - cur;
>
> - if (numAvailFull > temp)
>
> - numAvailFull = temp;
>
> - }
>
> -
>
> if (numAvailFull < 2)
>
> continue;
>
> numAvail = (numAvailFull <= p->numFastBytes ? numAvailFull :
> p->numFastBytes);
>
> @@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
>
>
> // ---------- LIT : REP_0 ----------
>
>
>
> - if (
>
> - // litPrice != 0 && // 18.new
>
> - !nextIsLit
>
> + if (!nextIsLit
>
> + && litPrice != 0 // 18.new
>
> && matchByte != curByte
>
> && numAvailFull > 2)
>
> {
>
> @@ -1465,8 +1574,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> for (len = 3; len < limit && data[len] == data2[len]; len++)
>
> - {
>
> - }
>
> + {}
>
>
>
> {
>
> unsigned state2 = kLiteralNextStates[state];
>
> @@ -1474,8 +1582,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
>
> {
>
> unsigned offset = cur + len;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
>
>
> // do
>
> {
>
> @@ -1483,14 +1592,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> COptimal *opt;
>
> len--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = 0;
>
> opt->extra = 1;
>
> }
>
> @@ -1502,6 +1611,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> }
>
>
>
> startLen = 2; /* speed optimization */
>
> +
>
> {
>
> // ---------- REP ----------
>
> unsigned repIndex = 0; // 17.old
>
> @@ -1514,24 +1624,28 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
>
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
>
>
> // if (len < startLen) continue; // 18.new: speed optimization
>
>
>
> - while (last < cur + len)
>
> - p->opt[++last].price = kInfinityPrice;
>
> + {
>
> + unsigned offset = cur + len;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
> {
>
> unsigned len2 = len;
>
> price = repMatchPrice + GetPrice_PureRep(p, repIndex, state,
> posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)len2 - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState, len2);
>
> COptimal *opt = &p->opt[cur + len2];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> - opt->dist = repIndex;
>
> + opt->len = (UInt32)len2;
>
> + opt->dist = (UInt32)repIndex;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1551,15 +1665,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> -
>
> - len2 -= len;
>
> - if (len2 >= 3)
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> {
>
> unsigned state2 = kRepNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> - price +=
>
> - p->repLenEnc.prices[posState][(size_t)len - 2]
>
> + price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
>
> + GET_PRICE_0(p->isMatch[state2][posState2])
>
> + LitEnc_Matched_GetPrice(LIT_PROBS(position + len,
> data[(size_t)len - 1]),
>
> data[len], data2[len], p->ProbPrices);
>
> @@ -1570,31 +1683,40 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
>
>
> price += GetPrice_Rep_0(p, state2, posState2);
>
> +
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
> +
>
> + len2 -= len;
>
> + // if (len2 >= 3)
>
> + {
>
> {
>
> unsigned offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> - unsigned price2;
>
> + UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> - opt->dist = repIndex;
>
> + opt->dist = (UInt32)repIndex;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> }
>
> + }
>
> }
>
> }
>
> }
>
> @@ -1606,17 +1728,23 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> newLen = numAvail;
>
> for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
>
> - matches[numPairs] = newLen;
>
> + matches[numPairs] = (UInt32)newLen;
>
> numPairs += 2;
>
> }
>
>
>
> + // startLen = 2; /* speed optimization */
>
> +
>
> if (newLen >= startLen)
>
> {
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[state]);
>
> UInt32 dist;
>
> unsigned offs, posSlot, len;
>
> - while (last < cur + newLen)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + {
>
> + unsigned offset = cur + newLen;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
>
>
> offs = 0;
>
> while (startLen > matches[offs])
>
> @@ -1628,27 +1756,29 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (len = /*2*/ startLen; ; len++)
>
> {
>
> - UInt32 price = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc,
> posState, len);
>
> {
>
> COptimal *opt;
>
> - unsigned lenToPosState = len - 2; lenToPosState =
> GetLenToPosState2(lenToPosState);
>
> + unsigned lenNorm = len - 2;
>
> + lenNorm = GetLenToPosState2(lenNorm);
>
> if (dist < kNumFullDistances)
>
> - price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenNorm][dist &
> (kNumFullDistances - 1)];
>
> else
>
> - price += p->posSlotPrices[lenToPosState][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenNorm][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
>
>
> opt = &p->opt[cur + len];
>
> if (price < opt->price)
>
> {
>
> opt->price = price;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> }
>
>
>
> - if (/*_maxMode && */ len == matches[offs])
>
> + if (len == matches[offs])
>
> {
>
> + // if (p->_maxMode) {
>
> // MATCH : LIT : REP_0
>
>
>
> const Byte *data2 = data - dist - 1;
>
> @@ -1657,11 +1787,17 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> + {
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
>
>
> len2 -= len;
>
>
>
> - if (len2 >= 3)
>
> + // if (len2 >= 3)
>
> {
>
> unsigned state2 = kMatchNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> @@ -1677,27 +1813,30 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price += GetPrice_Rep_0(p, state2, posState2);
>
>
>
> offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> +
>
> + }
>
>
>
> offs += 2;
>
> if (offs == numPairs)
>
> @@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
> }
>
> +
>
> + do
>
> + p->opt[last].price = kInfinityPrice;
>
> + while (--last);
>
> +
>
> + return Backward(p, cur);
>
> }
>
>
>
>
>
> @@ -1735,6 +1880,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> p->backRes = MARK_LIT;
>
> if (numAvail < 2)
>
> return 1;
>
> + // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
>
> if (numAvail > LZMA_MATCH_LEN_MAX)
>
> numAvail = LZMA_MATCH_LEN_MAX;
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> @@ -1746,10 +1892,11 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> const Byte *data2 = data - p->reps[i];
>
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> if (len >= p->numFastBytes)
>
> {
>
> - p->backRes = i;
>
> + p->backRes = (UInt32)i;
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> }
>
> @@ -1793,7 +1940,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> || (repLen + 2 >= mainLen && mainDist >= (1 << 9))
>
> || (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
>
> {
>
> - p->backRes = repIndex;
>
> + p->backRes = (UInt32)repIndex;
>
> MOVE_POS(p, repLen - 1)
>
> return repLen;
>
> }
>
> @@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p,
> UInt32 nowPos)
> }
>
>
>
>
>
> -
>
> -static void FillAlignPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
>
> {
>
> unsigned i;
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> const CLzmaProb *probs = p->posAlignEncoder;
>
> - p->alignPriceCount = 0;
>
> + // p->alignPriceCount = 0;
>
> for (i = 0; i < kAlignTableSize / 2; i++)
>
> {
>
> UInt32 price = 0;
>
> - unsigned symbol = i;
>
> + unsigned sym = i;
>
> unsigned m = 1;
>
> unsigned bit;
>
> UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> prob = probs[m];
>
> p->alignPrices[i ] = price + GET_PRICEa_0(prob);
>
> p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
>
> @@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
> }
>
>
>
>
>
> -static void FillDistancesPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
>
> {
>
> + // int y; for (y = 0; y < 100; y++) {
>
> +
>
> UInt32 tempPrices[kNumFullDistances];
>
> - unsigned i, lenToPosState;
>
> + unsigned i, lps;
>
>
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> p->matchPriceCount = 0;
>
>
>
> - for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
>
> + for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
>
> {
>
> unsigned posSlot = GetPosSlot1(i);
>
> - unsigned footerBits = ((posSlot >> 1) - 1);
>
> + unsigned footerBits = (posSlot >> 1) - 1;
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> + const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
>
> // tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base,
> footerBits, i - base, p->ProbPrices);
>
> -
>
> - const CLzmaProb *probs = p->posEncoders + base;
>
> UInt32 price = 0;
>
> unsigned m = 1;
>
> - unsigned symbol = i - base;
>
> + unsigned sym = i;
>
> + unsigned offset = (unsigned)1 << footerBits;
>
> + base += i;
>
> +
>
> + if (footerBits)
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> price += GET_PRICEa(probs[m], bit);
>
> m = (m << 1) + bit;
>
> }
>
> while (--footerBits);
>
> - tempPrices[i] = price;
>
> +
>
> + {
>
> + unsigned prob = probs[m];
>
> + tempPrices[base ] = price + GET_PRICEa_0(prob);
>
> + tempPrices[base + offset] = price + GET_PRICEa_1(prob);
>
> + }
>
> }
>
>
>
> - for (lenToPosState = 0; lenToPosState < kNumLenToPosStates;
> lenToPosState++)
>
> + for (lps = 0; lps < kNumLenToPosStates; lps++)
>
> {
>
> - unsigned posSlot;
>
> - const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
>
> - UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
>
> - unsigned distTableSize = p->distTableSize;
>
> - const CLzmaProb *probs = encoder;
>
> - for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
>
> + unsigned slot;
>
> + unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
>
> + UInt32 *posSlotPrices = p->posSlotPrices[lps];
>
> + const CLzmaProb *probs = p->posSlotEncoder[lps];
>
> +
>
> + for (slot = 0; slot < distTableSize2; slot++)
>
> {
>
> - // posSlotPrices[posSlot] = RcTree_GetPrice(encoder,
> kNumPosSlotBits, posSlot, p->ProbPrices);
>
> - UInt32 price = 0;
>
> + // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits,
> slot, p->ProbPrices);
>
> + UInt32 price;
>
> unsigned bit;
>
> - unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
>
> - UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
>
> - posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
>
> - posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
>
> + unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
>
> + unsigned prob;
>
> + bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
>
> + posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
>
> + posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
>
> }
>
> - for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
>
> - posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> -
>
> +
>
> {
>
> - UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
>
> + UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> + for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
>
> {
>
> - distancesPrices[0] = posSlotPrices[0];
>
> - distancesPrices[1] = posSlotPrices[1];
>
> - distancesPrices[2] = posSlotPrices[2];
>
> - distancesPrices[3] = posSlotPrices[3];
>
> + posSlotPrices[(size_t)slot * 2 ] += delta;
>
> + posSlotPrices[(size_t)slot * 2 + 1] += delta;
>
> + delta += ((UInt32)1 << kNumBitPriceShiftBits);
>
> }
>
> + }
>
> +
>
> + {
>
> + UInt32 *dp = p->distancesPrices[lps];
>
> +
>
> + dp[0] = posSlotPrices[0];
>
> + dp[1] = posSlotPrices[1];
>
> + dp[2] = posSlotPrices[2];
>
> + dp[3] = posSlotPrices[3];
>
> +
>
> for (i = 4; i < kNumFullDistances; i += 2)
>
> {
>
> UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
>
> - distancesPrices[i ] = slotPrice + tempPrices[i];
>
> - distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
>
> + dp[i ] = slotPrice + tempPrices[i];
>
> + dp[i + 1] = slotPrice + tempPrices[i + 1];
>
> }
>
> }
>
> }
>
> + // }
>
> }
>
>
>
>
>
> @@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> if (len != 1)
>
> {
>
> LenEnc_Encode(&p->repLenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->repLenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->repLenEnc, posState,
> &p->repLenProbs, p->ProbPrices);
>
> -
>
> + --p->repLenEncCounter;
>
> p->state = kRepNextStates[p->state];
>
> }
>
> }
>
> @@ -2260,9 +2422,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> p->state = kMatchNextStates[p->state];
>
>
>
> LenEnc_Encode(&p->lenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->lenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->lenEnc, posState,
> &p->lenProbs, p->ProbPrices);
>
> + // --p->lenEnc.counter;
>
>
>
> dist -= LZMA_NUM_REPS;
>
> p->reps[3] = p->reps[2];
>
> @@ -2274,17 +2434,17 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> GetPosSlot(dist, posSlot);
>
> // RcTree_Encode_PosSlot(&p->rc,
> p->posSlotEncoder[GetLenToPosState(len)], posSlot);
>
> {
>
> - UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
>
> + UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
>
> range = p->rc.range;
>
> probs = p->posSlotEncoder[GetLenToPosState(len)];
>
> do
>
> {
>
> - CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
>
> - UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
>
> - symbol <<= 1;
>
> + CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
>
> + UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
>
> + sym <<= 1;
>
> RC_BIT(&p->rc, prob, bit);
>
> }
>
> - while (symbol < (1 << kNumPosSlotBits * 2));
>
> + while (sym < (1 << kNumPosSlotBits * 2));
>
> p->rc.range = range;
>
> }
>
>
>
> @@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> if (dist < kNumFullDistances)
>
> {
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> - RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, dist - base);
>
> + RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, (unsigned)(dist /* - base */));
>
> }
>
> else
>
> {
>
> @@ -2331,14 +2491,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> bit = dist & 1; dist >>= 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit); m = (m << 1) + bit;
>
> bit = dist & 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit);
>
> p->rc.range = range;
>
> - p->alignPriceCount++;
>
> + // p->alignPriceCount++;
>
> }
>
> }
>
> }
>
> }
>
> }
>
>
>
> - nowPos32 += len;
>
> + nowPos32 += (UInt32)len;
>
> p->additionalOffset -= len;
>
>
>
> if (p->additionalOffset == 0)
>
> @@ -2347,10 +2507,27 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
>
>
> if (!p->fastMode)
>
> {
>
> - if (p->matchPriceCount >= (1 << 7))
>
> + /*
>
> + if (p->alignPriceCount >= 16) // kAlignTableSize
>
> + FillAlignPrices(p);
>
> + if (p->matchPriceCount >= 128)
>
> FillDistancesPrices(p);
>
> - if (p->alignPriceCount >= kAlignTableSize)
>
> + if (p->lenEnc.counter <= 0)
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + */
>
> + if (p->matchPriceCount >= 64)
>
> + {
>
> FillAlignPrices(p);
>
> + // { int y; for (y = 0; y < 100; y++) {
>
> + FillDistancesPrices(p);
>
> + // }}
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + }
>
> + if (p->repLenEncCounter <= 0)
>
> + {
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> + LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb,
> &p->repLenProbs, p->ProbPrices);
>
> + }
>
> }
>
>
>
> if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
>
> @@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
>
>
> p->optEnd = 0;
>
> p->optCur = 0;
>
> +
>
> + {
>
> + for (i = 0; i < kNumOpts; i++)
>
> + p->opt[i].price = kInfinityPrice;
>
> + }
>
> +
>
> p->additionalOffset = 0;
>
>
>
> p->pbMask = (1 << p->pb) - 1;
>
> p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
>
> }
>
>
>
> +
>
> void LzmaEnc_InitPrices(CLzmaEnc *p)
>
> {
>
> if (!p->fastMode)
>
> @@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
> p->lenEnc.tableSize =
>
> p->repLenEnc.tableSize =
>
> p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
>
> +
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> +
>
> LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs,
> p->ProbPrices);
>
> LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs,
> p->ProbPrices);
>
> }
>
> @@ -2583,7 +2770,7 @@ typedef struct
> ISeqOutStream vt;
>
> Byte *data;
>
> SizeT rem;
>
> - Bool overflow;
>
> + BoolInt overflow;
>
> } CLzmaEnc_SeqOutStreamBuf;
>
>
>
> static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void
> *data, size_t size)
>
> @@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle
> pp)
> }
>
>
>
>
>
> -SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
>
> +SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
>
> Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32
> *unpackSize)
>
> {
>
> CLzmaEnc *p = (CLzmaEnc *)pp;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> index d7426d38a4..04f9d98e7d 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> @@ -1,6 +1,28 @@
> HISTORY of the LZMA SDK
>
> -----------------------
>
>
>
> +19.00 2019-02-21
>
> +-------------------------
>
> +- Encryption strength for 7z archives was increased:
>
> + the size of random initialization vector was increased from 64-bit to
> 128-bit,
>
> + and the pseudo-random number generator was improved.
>
> +- The bug in 7zIn.c code was fixed.
>
> +
>
> +
>
> +18.06 2018-12-30
>
> +-------------------------
>
> +- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
>
> + and there are minor changes in compression ratio.
>
> +- Some bugs were fixed.
>
> +- The bug in 7-Zip 18.02-18.05 was fixed:
>
> + There was memory leak in multithreading xz decoder -
> XzDecMt_Decode(),
>
> + if xz stream contains only one block.
>
> +- The changes for MSVS compiler makefiles:
>
> + - the makefiles now use "PLATFORM" macroname with values (x64, x86,
> arm64)
>
> + instead of "CPU" macroname with values (AMD64, ARM64).
>
> + - the makefiles by default now use static version of the run-time
library.
>
> +
>
> +
>
> 18.05 2018-04-30
>
> -------------------------
>
> - The speed for LZMA/LZMA2 compressing was increased
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> index a3deb20943..e17cd4fe1a 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> @@ -1,4 +1,4 @@
> -LZMA SDK 18.05
>
> +LZMA SDK 19.00
>
> --------------
>
>
>
> LZMA SDK provides the documentation, samples, header files,
>
> --
> 2.27.0.windows.1
[-- Attachment #2: Type: text/html, Size: 382972 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread* 回复: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
2020-12-20 13:33 ` Liu, WeiX C
@ 2020-12-21 1:10 ` gaoliming
2020-12-21 7:26 ` weix.c.liu
0 siblings, 1 reply; 7+ messages in thread
From: gaoliming @ 2020-12-21 1:10 UTC (permalink / raw)
To: 'Liu, WeiX C', devel; +Cc: 'Laszlo Ersek'
[-- Attachment #1: Type: text/plain, Size: 114115 bytes --]
Do you verify the boot functionality (boot to shell)? Do you collect the image size (old LZMA version and new LZMA version)?
Thanks
Liming
发件人: Liu, WeiX C <weix.c.liu@intel.com>
发送时间: 2020年12月20日 21:34
收件人: gaoliming <gaoliming@byosoft.com.cn>; devel@edk2.groups.io
抄送: 'Laszlo Ersek' <lersek@redhat.com>
主题: RE: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
Hi liming,
I listed unittest result as below with Ovmf build, cause I have no apple machine so only list VS、GCC and clang tool chain .
VS2019
GCC5
CLANGPDB
IA32
Pass
Pass
Pass
X64
Pass
Pass
Pass
IA32x64
Pass
Pass
Pass
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >
Sent: 2020年12月17日 9:30
To: Liu, WeiX C <weix.c.liu@intel.com <mailto:weix.c.liu@intel.com> >; devel@edk2.groups.io <mailto:devel@edk2.groups.io>
Cc: 'Laszlo Ersek' <lersek@redhat.com <mailto:lersek@redhat.com> >
Subject: 回复: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
Wei:
Can you list the unit test for this patch? Which tool chain and arch have
been verified?
Because LZMA is well used, I request to verify it on VS, GCC, XCODE and
CLANG tool chain for the different ARCHs.
Thanks
Liming
> -----邮件原件-----
> 发件人: Wei Liu <weix.c.liu@intel.com <mailto:weix.c.liu@intel.com> >
> 发送时间: 2020年12月15日 10:07
> 收件人: devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> 抄送: Wei Liu <weix.c.liu@intel.com <mailto:weix.c.liu@intel.com> >; Liming Gao
> <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >; Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com> >
> 主题: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00
> version
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
> New formal release in https://www.7-zip.org/sdk.html is 19.00.
>
> Cc: Liming Gao <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >
>
> Cc: Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com> >
> Signed-off-by: Wei Liu <weix.c.liu@intel.com <mailto:weix.c.liu@intel.com> >
> ---
> BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4
> ++--
> BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5
> +++--
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9
> +++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 164
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++++++++++++++++++++++++++++++++++++++--------------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14
> +++++++-------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32
> ++++++++++++++++----------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 733
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
++++++++++++++++++++++++++--------------------------------------------------
----
>
----------------------------------------------------------------------------
-----------------
>
----------------------------------------------------------------------------
-----------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 22
> ++++++++++++++++++++++
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
> 12 files changed, 710 insertions(+), 408 deletions(-)
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> index 7cf967a774..b2338e2598 100644
> --- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> +++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> @@ -1,3 +1,3 @@
> -LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
>
> -was placed in the public domain on 2018-04-30. It was
>
> +LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
>
> +was placed in the public domain on 2019-02-21. It was
>
> released on the http://www.7-zip.org/sdk.html website.
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> index bebdb9aa84..bf4706ee23 100644
> --- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> +++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> @@ -1,9 +1,9 @@
> /** @file
>
> LZMA Compress/Decompress tool (LzmaCompress)
>
>
>
> - Based on LZMA SDK 18.05:
>
> + Based on LZMA SDK 19.00:
>
> LzmaUtil.c -- Test application for LZMA compression
>
> - 2018-04-30 : Igor Pavlov : Public domain
>
> + 2019-02-21 : Igor Pavlov : Public domain
>
>
>
> Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
>
> @@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate
> memory";
> const char *kDataErrorMessage = "Data error";
>
> const char *kInvalidParamValMessage = "Invalid parameter value";
>
>
>
> -static Bool mQuietMode = False;
>
> +static BoolInt mQuietMode = False;
>
> static CONVERTER_TYPE mConType = NoConverter;
>
>
>
> UINT64 mDictionarySize = 28;
>
> @@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
> CFileOutStream outStream;
>
> int res;
>
> int encodeMode = 0;
>
> - Bool modeWasSet = False;
>
> + BoolInt modeWasSet = False;
>
> const char *inputFile = NULL;
>
> const char *outputFile = "file.tmp";
>
> int param;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> index 4977cdaa66..593f5aa259 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> @@ -1,5 +1,5 @@
> /* 7zTypes.h -- Basic types
>
> -2017-07-17 : Igor Pavlov : Public domain */
>
> +2018-08-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __7Z_TYPES_H
>
> #define __7Z_TYPES_H
>
> @@ -103,7 +103,8 @@ typedef UInt32 SizeT;
> typedef size_t SizeT;
>
> #endif
>
>
>
> -typedef int Bool;
>
> +typedef int BoolInt;
>
> +/* typedef BoolInt Bool; */
>
> #define True 1
>
> #define False 0
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> index ed3aa94270..0074c64be9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> @@ -1,7 +1,7 @@
> -#define MY_VER_MAJOR 18
>
> -#define MY_VER_MINOR 05
>
> +#define MY_VER_MAJOR 19
>
> +#define MY_VER_MINOR 00
>
> #define MY_VER_BUILD 0
>
> -#define MY_VERSION_NUMBERS "18.05"
>
> +#define MY_VERSION_NUMBERS "19.00"
>
> #define MY_VERSION MY_VERSION_NUMBERS
>
>
>
> #ifdef MY_CPU_NAME
>
> @@ -10,7 +10,7 @@
> #define MY_VERSION_CPU MY_VERSION
>
> #endif
>
>
>
> -#define MY_DATE "2018-04-30"
>
> +#define MY_DATE "2019-02-21"
>
> #undef MY_COPYRIGHT
>
> #undef MY_VERSION_COPYRIGHT_DATE
>
> #define MY_AUTHOR_NAME "Igor Pavlov"
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> index 7fb27282c7..5f74c1c0cb 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> @@ -1,5 +1,5 @@
> /* CpuArch.h -- CPU specific code
>
> -2017-09-04 : Igor Pavlov : Public domain */
>
> +2018-02-18 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __CPU_ARCH_H
>
> #define __CPU_ARCH_H
>
> @@ -318,15 +318,16 @@ enum
>
>
> void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32
*d);
>
>
>
> -Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> +BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> int x86cpuid_GetFirm(const Cx86cpuid *p);
>
>
>
> #define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) &
0xF))
>
> #define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) &
> 0xF))
>
> #define x86cpuid_GetStepping(ver) (ver & 0xF)
>
>
>
> -Bool CPU_Is_InOrder();
>
> -Bool CPU_Is_Aes_Supported();
>
> +BoolInt CPU_Is_InOrder();
>
> +BoolInt CPU_Is_Aes_Supported();
>
> +BoolInt CPU_IsSupported_PageGB();
>
>
>
> #endif
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> index 6ea82a9b53..4eefc17dd2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> @@ -1,5 +1,5 @@
> /* LzFind.c -- Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-07-08 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -138,7 +138,7 @@ static void
> MatchFinder_SetDefaultSettings(CMatchFinder *p)
>
>
> void MatchFinder_Construct(CMatchFinder *p)
>
> {
>
> - UInt32 i;
>
> + unsigned i;
>
> p->bufferBase = NULL;
>
> p->directInput = 0;
>
> p->hash = NULL;
>
> @@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
>
>
> for (i = 0; i < 256; i++)
>
> {
>
> - UInt32 r = i;
>
> + UInt32 r = (UInt32)i;
>
> unsigned j;
>
> for (j = 0; j < 8; j++)
>
> r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
>
> @@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
> MatchFinder_ReduceOffsets(p, subValue);
>
> }
>
>
>
> +
>
> +MY_NO_INLINE
>
> static void MatchFinder_CheckLimits(CMatchFinder *p)
>
> {
>
> if (p->pos == kMaxValForNormalize)
>
> @@ -379,10 +381,16 @@ static void
> MatchFinder_CheckLimits(CMatchFinder *p)
> MatchFinder_SetLimits(p);
>
> }
>
>
>
> -static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> +
>
> +/*
>
> + (lenLimit > maxLen)
>
> +*/
>
> +MY_FORCE_INLINE
>
> +static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> - UInt32 *distances, UInt32 maxLen)
>
> + UInt32 *distances, unsigned maxLen)
>
> {
>
> + /*
>
> son[_cyclicBufferPos] = curMatch;
>
> for (;;)
>
> {
>
> @@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos,
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> return distances;
>
> @@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32
> lenLimit, UInt32 curMatch, UInt32 pos,
> }
>
> }
>
> }
>
> + */
>
> +
>
> + const Byte *lim = cur + lenLimit;
>
> + son[_cyclicBufferPos] = curMatch;
>
> + do
>
> + {
>
> + UInt32 delta = pos - curMatch;
>
> + if (delta >= _cyclicBufferSize)
>
> + break;
>
> + {
>
> + ptrdiff_t diff;
>
> + curMatch = son[_cyclicBufferPos - delta + ((delta >
_cyclicBufferPos) ?
> _cyclicBufferSize : 0)];
>
> + diff = (ptrdiff_t)0 - delta;
>
> + if (cur[maxLen] == cur[maxLen + diff])
>
> + {
>
> + const Byte *c = cur;
>
> + while (*c == c[diff])
>
> + {
>
> + if (++c == lim)
>
> + {
>
> + distances[0] = (UInt32)(lim - cur);
>
> + distances[1] = delta - 1;
>
> + return distances + 2;
>
> + }
>
> + }
>
> + {
>
> + unsigned len = (unsigned)(c - cur);
>
> + if (maxLen < len)
>
> + {
>
> + maxLen = len;
>
> + distances[0] = (UInt32)len;
>
> + distances[1] = delta - 1;
>
> + distances += 2;
>
> + }
>
> + }
>
> + }
>
> + }
>
> + }
>
> + while (--cutValue);
>
> +
>
> + return distances;
>
> }
>
>
>
> +
>
> +MY_FORCE_INLINE
>
> UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> UInt32 *distances, UInt32 maxLen)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> return distances;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = pair[0];
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const Byt
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = (UInt32)len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> + *ptr1 = pair0;
>
> *ptr0 = pair[1];
>
> return distances;
>
> }
>
> @@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const
> return;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> if (pb[len] == cur[len])
>
> {
>
> while (++len != lenLimit)
>
> @@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const
> p->buffer++; \
>
> if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
>
>
>
> -#define MOVE_POS_RET MOVE_POS return offset;
>
> +#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
>
>
>
> static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
>
>
>
> #define GET_MATCHES_HEADER2(minLen, ret_op) \
>
> - UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> - lenLimit = p->lenLimit; { if (lenLimit < minLen) {
MatchFinder_MovePos(p);
> ret_op; }} \
>
> + unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> + lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen)
> { MatchFinder_MovePos(p); ret_op; }} \
>
> cur = p->buffer;
>
>
>
> #define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen,
> return 0)
>
> @@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder
> *p) { MOVE_POS; }
> #define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos,
> p->cyclicBufferSize, p->cutValue
>
>
>
> #define GET_MATCHES_FOOTER(offset, maxLen) \
>
> - offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
>
> - distances + offset, maxLen) - distances); MOVE_POS_RET;
>
> + offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch,
> MF_PARAMS(p), \
>
> + distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
>
>
>
> #define SKIP_FOOTER \
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
> MOVE_POS;
>
>
>
> #define UPDATE_maxLen { \
>
> ptrdiff_t diff = (ptrdiff_t)0 - d2; \
>
> const Byte *c = cur + maxLen; \
>
> const Byte *lim = cur + lenLimit; \
>
> for (; c != lim; c++) if (*(c + diff) != *c) break; \
>
> - maxLen = (UInt32)(c - cur); }
>
> + maxLen = (unsigned)(c - cur); }
>
>
>
> static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(2)
>
> HASH2_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -561,7 +615,7 @@ static UInt32
> Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -572,7 +626,8 @@ UInt32
> Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, d2, maxLen, offset, pos;
>
> + UInt32 h2, d2, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(3)
>
>
>
> @@ -594,12 +649,12 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> UPDATE_maxLen
>
> - distances[0] = maxLen;
>
> + distances[0] = (UInt32)maxLen;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -609,7 +664,8 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -618,12 +674,12 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
>
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -632,7 +688,8 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -648,10 +705,10 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -674,13 +731,13 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -741,7 +798,8 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -750,12 +808,11 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> -
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -764,7 +821,8 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -780,7 +838,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -791,7 +849,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (maxLen < 3)
>
> maxLen = 3;
>
>
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances + offset, maxLen) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -808,13 +866,13 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -877,12 +935,12 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> p->hash[hv] = p->pos;
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances, 2) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> SKIP_FOOTER
>
> @@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> @@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> index 2563824fcd..df32146f92 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> @@ -1,5 +1,5 @@
> /* LzFindMt.c -- multithreaded Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-12-29 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -232,38 +232,57 @@ static void
> MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
>
>
> #define kEmptyHashValue 0
>
>
>
> -/* #define MFMT_GM_INLINE */
>
> +#define MFMT_GM_INLINE
>
>
>
> #ifdef MFMT_GM_INLINE
>
>
>
> -#define NO_INLINE MY_FAST_CALL
>
> +/*
>
> + we use size_t for _cyclicBufferPos instead of UInt32
>
> + to eliminate "movsx" BUG in old MSVC x64 compiler.
>
> +*/
>
>
>
> -static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> - UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> - UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit,
> UInt32 size, UInt32 *posRes)
>
> +MY_NO_INLINE
>
> +static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte
> *cur, CLzRef *son,
>
> + size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> + UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32
> *limit, UInt32 size, UInt32 *posRes)
>
> {
>
> do
>
> {
>
> - UInt32 *distances = _distances + 1;
>
> - UInt32 curMatch = pos - *hash++;
>
> + UInt32 *_distances = ++distances;
>
> + UInt32 delta = *hash++;
>
>
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> UInt32 cutValue = _cutValue;
>
> - UInt32 maxLen = _maxLen;
>
> - for (;;)
>
> + unsigned maxLen = (unsigned)_maxLen;
>
> +
>
> + /*
>
> + if (size > 1)
>
> {
>
> - UInt32 delta = pos - curMatch;
>
> - if (cutValue-- == 0 || delta >= _cyclicBufferSize)
>
> + UInt32 delta = *hash;
>
> + if (delta < _cyclicBufferSize)
>
> {
>
> - *ptr0 = *ptr1 = kEmptyHashValue;
>
> - break;
>
> + UInt32 cyc1 = _cyclicBufferPos + 1;
>
> + CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ?
> _cyclicBufferSize : 0)) << 1);
>
> + Byte b = *(cur + 1 - delta);
>
> + _distances[0] = pair[0];
>
> + _distances[1] = b;
>
> }
>
> + }
>
> + */
>
> + if (cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + }
>
> + else
>
> + for(;;)
>
> + {
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta +
> ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = *pair;
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32
> lenLimit, UInt32 pos, const Byte *
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> - *ptr0 = pair[1];
>
> + UInt32 pair1 = pair[1];
>
> + *ptr1 = pair0;
>
> + *ptr0 = pair1;
>
> break;
>
> }
>
> }
>
> }
>
> - if (pb[len] < cur[len])
>
> - {
>
> - *ptr1 = curMatch;
>
> - ptr1 = pair + 1;
>
> - curMatch = *ptr1;
>
> - len1 = len;
>
> - }
>
> - else
>
> {
>
> - *ptr0 = curMatch;
>
> - ptr0 = pair;
>
> - curMatch = *ptr0;
>
> - len0 = len;
>
> + UInt32 curMatch = pos - delta;
>
> + // delta = pos - *pair;
>
> + // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >>
31];
>
> + if (pb[len] < cur[len])
>
> + {
>
> + delta = pos - pair[1];
>
> + *ptr1 = curMatch;
>
> + ptr1 = pair + 1;
>
> + len1 = len;
>
> + }
>
> + else
>
> + {
>
> + delta = pos - *pair;
>
> + *ptr0 = curMatch;
>
> + ptr0 = pair;
>
> + len0 = len;
>
> + }
>
> }
>
> }
>
> + if (--cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + break;
>
> + }
>
> }
>
> pos++;
>
> _cyclicBufferPos++;
>
> cur++;
>
> {
>
> UInt32 num = (UInt32)(distances - _distances);
>
> - *_distances = num - 1;
>
> - _distances += num;
>
> - limit -= num;
>
> + _distances[-1] = num;
>
> }
>
> }
>
> - while (limit > 0 && --size != 0);
>
> + while (distances < limit && --size != 0);
>
> *posRes = pos;
>
> - return limit;
>
> + return distances;
>
> }
>
>
>
> #endif
>
>
>
> +
>
> +
>
> static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
>
> {
>
> UInt32 numProcessed = 0;
>
> UInt32 curPos = 2;
>
> - UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
>
> + UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
>
>
>
> distances[1] = p->hashNumAvail;
>
>
>
> @@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p,
> UInt32 *distances)
> #else
>
> {
>
> UInt32 posRes;
>
> - curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer,
p->son,
> cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> - distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
>
> + curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer,
> p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> + distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos,
>
> + distances + limit,
>
> + size, &posRes) - distances);
>
> p->hashBufPos += posRes - pos;
>
> cyclicBufferPos += posRes - pos;
>
> p->buffer += posRes - pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> index 3d86c788f3..fdd17008c2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> @@ -1,5 +1,5 @@
> /* LzFindMt.h -- multithreaded Match finder for LZ algorithms
>
> -2017-04-03 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __LZ_FIND_MT_H
>
> #define __LZ_FIND_MT_H
>
> @@ -19,10 +19,10 @@ EXTERN_C_BEGIN
>
>
> typedef struct _CMtSync
>
> {
>
> - Bool wasCreated;
>
> - Bool needStart;
>
> - Bool exit;
>
> - Bool stopWriting;
>
> + BoolInt wasCreated;
>
> + BoolInt needStart;
>
> + BoolInt exit;
>
> + BoolInt stopWriting;
>
>
>
> CThread thread;
>
> CAutoResetEvent canStart;
>
> @@ -30,8 +30,8 @@ typedef struct _CMtSync
> CAutoResetEvent wasStopped;
>
> CSemaphore freeSemaphore;
>
> CSemaphore filledSemaphore;
>
> - Bool csWasInitialized;
>
> - Bool csWasEntered;
>
> + BoolInt csWasInitialized;
>
> + BoolInt csWasEntered;
>
> CCriticalSection cs;
>
> UInt32 numProcessedBlocks;
>
> } CMtSync;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> index 962b94bb63..4d1576419f 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> @@ -1,13 +1,13 @@
> /* LzmaDec.c -- LZMA Decoder
>
> -2018-02-28 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> +#include <string.h>
>
> +
>
> /* #include "CpuArch.h" */
>
> #include "LzmaDec.h"
>
>
>
> -#include <string.h>
>
> -
>
> #define kNumTopBits 24
>
> #define kTopValue ((UInt32)1 << kNumTopBits)
>
>
>
> @@ -19,7 +19,7 @@
>
>
> #define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code <<
8)
> | (*buf++); }
>
>
>
> -#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt +
> ((kBitModelTotal - ttt) >> kNumMoveBits));
>
> #define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt
> - (ttt >> kNumMoveBits));
>
> #define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
>
> @@ -66,7 +66,7 @@
>
>
> #define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit)
> return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
>
>
>
> -#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0_CHECK range = bound;
>
> #define UPDATE_1_CHECK range -= bound; code -= bound;
>
> #define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
>
> @@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> curLen = ((rem < len) ? (unsigned)rem : len);
>
> pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
>
>
>
> - processedPos += curLen;
>
> + processedPos += (UInt32)curLen;
>
>
>
> len -= curLen;
>
> if (curLen <= dicBufSize - pos)
>
> @@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> Byte *dest = dic + dicPos;
>
> ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
>
> const Byte *lim = dest + curLen;
>
> - dicPos += curLen;
>
> + dicPos += (SizeT)curLen;
>
> do
>
> *(dest) = (Byte)*(dest + src);
>
> while (++dest != lim);
>
> @@ -572,14 +572,14 @@ int MY_FAST_CALL
> LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
> p->buf = buf;
>
> p->range = range;
>
> p->code = code;
>
> - p->remainLen = len;
>
> + p->remainLen = (UInt32)len;
>
> p->dicPos = dicPos;
>
> p->processedPos = processedPos;
>
> p->reps[0] = rep0;
>
> p->reps[1] = rep1;
>
> p->reps[2] = rep2;
>
> p->reps[3] = rep3;
>
> - p->state = state;
>
> + p->state = (UInt32)state;
>
>
>
> return SZ_OK;
>
> }
>
> @@ -601,8 +601,8 @@ static void MY_FAST_CALL
> LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
> if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
>
> p->checkDicSize = p->prop.dicSize;
>
>
>
> - p->processedPos += len;
>
> - p->remainLen -= len;
>
> + p->processedPos += (UInt32)len;
>
> + p->remainLen -= (UInt32)len;
>
> while (len != 0)
>
> {
>
> len--;
>
> @@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const
> CLzmaDec *p, const Byte *buf, SizeT inS
> }
>
>
>
>
>
> -void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
>
> +void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt
> initState)
>
> {
>
> p->remainLen = kMatchSpecLenStart + 1;
>
> p->tempBufSize = 0;
>
> @@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> p->tempBufSize = rem;
>
> if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
>
> {
>
> - int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
>
> + int dummyRes = LzmaDec_TryDummy(p, p->tempBuf,
> (SizeT)rem);
>
> if (dummyRes == DUMMY_ERROR)
>
> {
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> *status = LZMA_STATUS_NEEDS_MORE_INPUT;
>
> return SZ_OK;
>
> }
>
> @@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> return SZ_ERROR_FAIL; /* some internal error */
>
> lookAhead -= rem;
>
> }
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> src += lookAhead;
>
> - inSize -= lookAhead;
>
> + inSize -= (SizeT)lookAhead;
>
> p->tempBufSize = 0;
>
> }
>
> }
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> index e281716fee..14086fc4f9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> @@ -1,5 +1,5 @@
> /* LzmaEnc.c -- LZMA Encoder
>
> -2018-04-29 : Igor Pavlov : Public domain */
>
> +2019-01-10: Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
> #define kNumBitPriceShiftBits 4
>
> #define kBitPrice (1 << kNumBitPriceShiftBits)
>
>
>
> +#define REP_LEN_COUNT 64
>
> +
>
> void LzmaEncProps_Init(CLzmaEncProps *p)
>
> {
>
> p->level = 5;
>
> @@ -183,8 +185,11 @@ typedef struct
> } COptimal;
>
>
>
>
>
> -#define kNumOpts (1 << 12)
>
> -#define kPackReserve (1 + kNumOpts * 2)
>
> +// 18.06
>
> +#define kNumOpts (1 << 11)
>
> +#define kPackReserve (kNumOpts * 8)
>
> +// #define kNumOpts (1 << 12)
>
> +// #define kPackReserve (1 + kNumOpts * 2)
>
>
>
> #define kNumLenToPosStates 4
>
> #define kNumPosSlotBits 6
>
> @@ -236,10 +241,18 @@ typedef struct
> typedef struct
>
> {
>
> unsigned tableSize;
>
> - unsigned counters[LZMA_NUM_PB_STATES_MAX];
>
> UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
>
> + // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols *
> 2];
>
> + // UInt32 prices2[kLenNumSymbolsTotal];
>
> } CLenPriceEnc;
>
>
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
>
> +
>
> +/*
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) -
2) &
> (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >>
9)))
>
> +*/
>
>
>
> typedef struct
>
> {
>
> @@ -308,16 +321,18 @@ typedef struct
> unsigned lc, lp, pb;
>
> unsigned lclp;
>
>
>
> - Bool fastMode;
>
> - Bool writeEndMark;
>
> - Bool finished;
>
> - Bool multiThread;
>
> - Bool needInit;
>
> + BoolInt fastMode;
>
> + BoolInt writeEndMark;
>
> + BoolInt finished;
>
> + BoolInt multiThread;
>
> + BoolInt needInit;
>
> + // BoolInt _maxMode;
>
>
>
> UInt64 nowPos64;
>
>
>
> unsigned matchPriceCount;
>
> - unsigned alignPriceCount;
>
> + // unsigned alignPriceCount;
>
> + int repLenEncCounter;
>
>
>
> unsigned distTableSize;
>
>
>
> @@ -325,7 +340,7 @@ typedef struct
> SRes result;
>
>
>
> #ifndef _7ZIP_ST
>
> - Bool mtMode;
>
> + BoolInt mtMode;
>
> // begin of CMatchFinderMt is used in LZ thread
>
> CMatchFinderMt matchFinderMt;
>
> // end of CMatchFinderMt is used in BT and HASH threads
>
> @@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const
> CLzmaEncProps *props2)
> p->lp = props.lp;
>
> p->pb = props.pb;
>
> p->fastMode = (props.algo == 0);
>
> + // p->_maxMode = True;
>
> p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
>
> {
>
> unsigned numHashBytes = 4;
>
> @@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #ifdef _LZMA_ENC_USE_BRANCH
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> RC_BIT_PRE(p, prob) \
>
> - if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> + if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> else { (p)->low += newBound; range -= newBound; ttt -= ttt >>
> kNumMoveBits; } \
>
> *(prob) = (CLzmaProb)ttt; \
>
> RC_NORM(p) \
>
> @@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #else
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> UInt32 mask; \
>
> RC_BIT_PRE(p, prob) \
>
> - mask = 0 - (UInt32)symbol; \
>
> + mask = 0 - (UInt32)bit; \
>
> range &= mask; \
>
> mask &= newBound; \
>
> range -= mask; \
>
> (p)->low += mask; \
>
> - mask = (UInt32)symbol - 1; \
>
> + mask = (UInt32)bit - 1; \
>
> range += newBound & mask; \
>
> mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
>
> mask += ((1 << kNumMoveBits) - 1); \
>
> @@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p,
> CLzmaProb *prob)
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
>
> +static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
>
> {
>
> UInt32 range = p->range;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
>
> - CLzmaProb *prob = probs + (symbol >> 8);
>
> - UInt32 bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> + // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
>
> + CLzmaProb *prob = probs + (sym >> 8);
>
> + UInt32 bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte)
>
> +static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs,
> UInt32 sym, UInt32 matchByte)
>
> {
>
> UInt32 range = p->range;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> CLzmaProb *prob;
>
> UInt32 bit;
>
> matchByte <<= 1;
>
> - // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol
>>
> 8)), (symbol >> 7) & 1);
>
> - prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
>
> - bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >>
> 8)), (sym >> 7) & 1);
>
> + prob = probs + (offs + (matchByte & offs) + (sym >> 8));
>
> + bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> @@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> }
>
>
>
>
>
> -#define GET_PRICE(prob, symbol) \
>
> - p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICE(prob, bit) \
>
> + p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> -#define GET_PRICEa(prob, symbol) \
>
> - ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICEa(prob, bit) \
>
> + ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> #define GET_PRICE_0(prob) p->ProbPrices[(prob) >>
> kNumMoveReducingBits]
>
> #define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1))
>>
> kNumMoveReducingBits]
>
> @@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> #define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >>
> kNumMoveReducingBits]
>
>
>
>
>
> -static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol,
const
> CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const
> CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> - price += GET_PRICEa(probs[symbol], bit);
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> }
>
> - while (symbol >= 2);
>
> + while (sym >= 2);
>
> return price;
>
> }
>
>
>
>
>
> -static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> sym, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> matchByte <<= 1;
>
> - price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)],
> (symbol >> 7) & 1);
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)],
> (sym >> 7) & 1);
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> return price;
>
> }
>
>
>
>
>
> -static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, UInt32 symbol)
>
> +static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, unsigned sym)
>
> {
>
> UInt32 range = rc->range;
>
> unsigned m = 1;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - unsigned bit = symbol & 1;
>
> + unsigned bit = sym & 1;
>
> // RangeEnc_EncodeBit(rc, probs + m, bit);
>
> - symbol >>= 1;
>
> + sym >>= 1;
>
> RC_BIT(rc, probs + m, bit);
>
> m = (m << 1) | bit;
>
> }
>
> @@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
> p->high[i] = kProbInitValue;
>
> }
>
>
>
> -static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol,
> unsigned posState)
>
> +static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym,
> unsigned posState)
>
> {
>
> UInt32 range, ttt, newBound;
>
> CLzmaProb *probs = p->low;
>
> range = rc->range;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols)
>
> + if (sym >= kLenNumLowSymbols)
>
> {
>
> RC_BIT_1(rc, probs);
>
> probs += kLenNumLowSymbols;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols * 2)
>
> + if (sym >= kLenNumLowSymbols * 2)
>
> {
>
> RC_BIT_1(rc, probs);
>
> rc->range = range;
>
> - // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol -
> kLenNumLowSymbols * 2);
>
> - LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
>
> + // RcTree_Encode(rc, p->high, kLenNumHighBits, sym -
> kLenNumLowSymbols * 2);
>
> + LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
>
> return;
>
> }
>
> - symbol -= kLenNumLowSymbols;
>
> + sym -= kLenNumLowSymbols;
>
> }
>
>
>
> - // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, symbol);
>
> + // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, sym);
>
> {
>
> unsigned m;
>
> unsigned bit;
>
> RC_BIT_0(rc, probs);
>
> probs += (posState << (1 + kLenNumLowBits));
>
> - bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) +
bit;
>
> - bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) +
bit;
>
> - bit = symbol & 1; RC_BIT(rc, probs + m, bit);
>
> + bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
>
> + bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
>
> + bit = sym & 1; RC_BIT(rc, probs + m, bit);
>
> rc->range = range;
>
> }
>
> }
>
> @@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs,
> UInt32 startPrice, UInt32 *price
> }
>
>
>
>
>
> -MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
>
> - CLenPriceEnc *p, unsigned posState,
>
> +MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
>
> + CLenPriceEnc *p,
>
> + unsigned numPosStates,
>
> const CLenEnc *enc,
>
> const CProbPrice *ProbPrices)
>
> {
>
> - // int y; for (y = 0; y < 100; y++) {
>
> - UInt32 a;
>
> - unsigned i, numSymbols;
>
> + UInt32 b;
>
> +
>
> + {
>
> + unsigned prob = enc->low[0];
>
> + UInt32 a, c;
>
> + unsigned posState;
>
> + b = GET_PRICEa_1(prob);
>
> + a = GET_PRICEa_0(prob);
>
> + c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (posState = 0; posState < numPosStates; posState++)
>
> + {
>
> + UInt32 *prices = p->prices[posState];
>
> + const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> + SetPrices_3(probs, a, prices, ProbPrices);
>
> + SetPrices_3(probs + kLenNumLowSymbols, c, prices +
> kLenNumLowSymbols, ProbPrices);
>
> + }
>
> + }
>
>
>
> - UInt32 *prices = p->prices[posState];
>
> + /*
>
> {
>
> - const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> - SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
>
> + unsigned i;
>
> + UInt32 b;
>
> + a = GET_PRICEa_0(enc->low[0]);
>
> + for (i = 0; i < kLenNumLowSymbols; i++)
>
> + p->prices2[i] = a;
>
> a = GET_PRICEa_1(enc->low[0]);
>
> - SetPrices_3(probs + kLenNumLowSymbols, a +
> GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices +
> kLenNumLowSymbols, ProbPrices);
>
> + b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
>
> + p->prices2[i] = b;
>
> a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> }
>
> - numSymbols = p->tableSize;
>
> - p->counters[posState] = numSymbols;
>
> - for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
>
> + */
>
> +
>
> + // p->counter = numSymbols;
>
> + // p->counter = 64;
>
> +
>
> {
>
> - prices[i] = a +
>
> - // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> - LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2,
> ProbPrices);
>
> - /*
>
> - unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
>
> - UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> - UInt32 prob = enc->high[(1 << 7) + sym];
>
> - prices[i ] = price + GET_PRICEa_0(prob);
>
> - prices[i + 1] = price + GET_PRICEa_1(prob);
>
> - */
>
> - }
>
> - // }
>
> -}
>
> + unsigned i = p->tableSize;
>
> +
>
> + if (i > kLenNumLowSymbols * 2)
>
> + {
>
> + const CLzmaProb *probs = enc->high;
>
> + UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
>
> + i -= kLenNumLowSymbols * 2 - 1;
>
> + i >>= 1;
>
> + b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> + do
>
> + {
>
> + /*
>
> + p->prices2[i] = a +
>
> + // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> + LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
>
> + */
>
> + // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> + unsigned sym = --i + (1 << (kLenNumHighBits - 1));
>
> + UInt32 price = b;
>
> + do
>
> + {
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> + }
>
> + while (sym >= 2);
>
>
>
> -static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned
> numPosStates,
>
> - const CLenEnc *enc,
>
> - const CProbPrice *ProbPrices)
>
> -{
>
> - unsigned posState;
>
> - for (posState = 0; posState < numPosStates; posState++)
>
> - LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
>
> -}
>
> + {
>
> + unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits -
1))];
>
> + prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
>
> + prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
>
> + }
>
> + }
>
> + while (i);
>
>
>
> + {
>
> + unsigned posState;
>
> + size_t num = (p->tableSize - kLenNumLowSymbols * 2) *
> sizeof(p->prices[0][0]);
>
> + for (posState = 1; posState < numPosStates; posState++)
>
> + memcpy(p->prices[posState] + kLenNumLowSymbols * 2,
> p->prices[0] + kLenNumLowSymbols * 2, num);
>
> + }
>
> + }
>
> + }
>
> +}
>
>
>
> /*
>
> #ifdef SHOW_STAT
>
> @@ -917,7 +976,7 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc
> *p, unsigned numPosStates,
>
>
> #define MOVE_POS(p, num) { \
>
> p->additionalOffset += (num); \
>
> - p->matchFinder.Skip(p->matchFinderObj, (num)); }
>
> + p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
>
>
>
>
>
> static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
>
> @@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> const Byte *p2 = p1 + len;
>
> ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
>
> const Byte *lim = p1 + numAvail;
>
> - for (; p2 != lim && *p2 == p2[dif]; p2++);
>
> + for (; p2 != lim && *p2 == p2[dif]; p2++)
>
> + {}
>
> return (unsigned)(p2 - p1);
>
> }
>
> }
>
> @@ -977,7 +1037,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> + GET_PRICE_1(p->isRep[state]) \
>
> + GET_PRICE_0(p->isRepG0[state])
>
>
>
> -
>
> +MY_FORCE_INLINE
>
> static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex,
> size_t state, size_t posState)
>
> {
>
> UInt32 price;
>
> @@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p,
> unsigned cur)
> for (;;)
>
> {
>
> UInt32 dist = p->opt[cur].dist;
>
> - UInt32 len = p->opt[cur].len;
>
> - UInt32 extra = p->opt[cur].extra;
>
> + unsigned len = (unsigned)p->opt[cur].len;
>
> + unsigned extra = (unsigned)p->opt[cur].extra;
>
> cur -= len;
>
>
>
> if (extra)
>
> {
>
> wr--;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> cur -= extra;
>
> len = extra;
>
> if (extra == 1)
>
> @@ -1045,7 +1105,7 @@ static unsigned Backward(CLzmaEnc *p, unsigned
> cur)
>
>
> wr--;
>
> p->opt[wr].dist = dist;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> }
>
> }
>
>
>
> @@ -1102,7 +1162,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> repLens[i] = 0;
>
> continue;
>
> }
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> repLens[i] = len;
>
> if (len > repLens[repMaxIndex])
>
> repMaxIndex = i;
>
> @@ -1111,7 +1172,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (repLens[repMaxIndex] >= p->numFastBytes)
>
> {
>
> unsigned len;
>
> - p->backRes = repMaxIndex;
>
> + p->backRes = (UInt32)repMaxIndex;
>
> len = repLens[repMaxIndex];
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> @@ -1128,8 +1189,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
> +
>
> + last = repLens[repMaxIndex];
>
> + if (last <= mainLen)
>
> + last = mainLen;
>
>
>
> - if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] <
> 2)
>
> + if (last < 2 && curByte != matchByte)
>
> {
>
> p->backRes = MARK_LIT;
>
> return 1;
>
> @@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> LitEnc_Matched_GetPrice(probs, curByte, matchByte,
> p->ProbPrices) :
>
> LitEnc_GetPrice(probs, curByte, p->ProbPrices));
>
> }
>
> -
>
> +
>
> MakeAs_Lit(&p->opt[1]);
>
>
>
> matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
>
>
>
> - if (matchByte == curByte)
>
> + // 18.06
>
> + if (matchByte == curByte && repLens[0] == 0)
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p,
> p->state, posState);
>
> if (shortRepPrice < p->opt[1].price)
>
> @@ -1160,16 +1226,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[1].price = shortRepPrice;
>
> MakeAs_ShortRep(&p->opt[1]);
>
> }
>
> + if (last < 2)
>
> + {
>
> + p->backRes = p->opt[1].dist;
>
> + return 1;
>
> + }
>
> }
>
> -
>
> - last = (mainLen >= repLens[repMaxIndex] ? mainLen :
> repLens[repMaxIndex]);
>
> -
>
> - if (last < 2)
>
> - {
>
> - p->backRes = p->opt[1].dist;
>
> - return 1;
>
> - }
>
> -
>
> +
>
> p->opt[1].len = 1;
>
>
>
> p->opt[0].reps[0] = reps[0];
>
> @@ -1177,13 +1240,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[0].reps[2] = reps[2];
>
> p->opt[0].reps[3] = reps[3];
>
>
>
> - {
>
> - unsigned len = last;
>
> - do
>
> - p->opt[len--].price = kInfinityPrice;
>
> - while (len >= 2);
>
> - }
>
> -
>
> // ---------- REP ----------
>
>
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> @@ -1195,13 +1251,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)repLen - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState,
> repLen);
>
> COptimal *opt = &p->opt[repLen];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = repLen;
>
> - opt->dist = i;
>
> + opt->len = (UInt32)repLen;
>
> + opt->dist = (UInt32)i;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1211,38 +1267,41 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> // ---------- MATCH ----------
>
> {
>
> - unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
>
> + unsigned len = repLens[0] + 1;
>
> if (len <= mainLen)
>
> {
>
> unsigned offs = 0;
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[p->state]);
>
>
>
> - while (len > matches[offs])
>
> - offs += 2;
>
> + if (len < 2)
>
> + len = 2;
>
> + else
>
> + while (len > matches[offs])
>
> + offs += 2;
>
>
>
> for (; ; len++)
>
> {
>
> COptimal *opt;
>
> UInt32 dist = matches[(size_t)offs + 1];
>
> - UInt32 price2 = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice +
> GET_PRICE_LEN(&p->lenEnc, posState, len);
>
> unsigned lenToPosState = GetLenToPosState(len);
>
>
>
> if (dist < kNumFullDistances)
>
> - price2 += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> else
>
> {
>
> unsigned slot;
>
> GetPosSlot2(dist, slot);
>
> - price2 += p->alignPrices[dist & kAlignMask];
>
> - price2 += p->posSlotPrices[lenToPosState][slot];
>
> + price += p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenToPosState][slot];
>
> }
>
>
>
> opt = &p->opt[len];
>
>
>
> - if (price2 < opt->price)
>
> + if (price < opt->price)
>
> {
>
> - opt->price = price2;
>
> - opt->len = len;
>
> + opt->price = price;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> @@ -1277,16 +1336,43 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (;;)
>
> {
>
> - UInt32 numAvail, numAvailFull;
>
> + unsigned numAvail;
>
> + UInt32 numAvailFull;
>
> unsigned newLen, numPairs, prev, state, posState, startLen;
>
> - UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
>
> - Bool nextIsLit;
>
> + UInt32 litPrice, matchPrice, repMatchPrice;
>
> + BoolInt nextIsLit;
>
> Byte curByte, matchByte;
>
> const Byte *data;
>
> COptimal *curOpt, *nextOpt;
>
>
>
> if (++cur == last)
>
> - return Backward(p, cur);
>
> + break;
>
> +
>
> + // 18.06
>
> + if (cur >= kNumOpts - 64)
>
> + {
>
> + unsigned j, best;
>
> + UInt32 price = p->opt[cur].price;
>
> + best = cur;
>
> + for (j = cur + 1; j <= last; j++)
>
> + {
>
> + UInt32 price2 = p->opt[j].price;
>
> + if (price >= price2)
>
> + {
>
> + price = price2;
>
> + best = j;
>
> + }
>
> + }
>
> + {
>
> + unsigned delta = best - cur;
>
> + if (delta != 0)
>
> + {
>
> + MOVE_POS(p, delta);
>
> + }
>
> + }
>
> + cur = best;
>
> + break;
>
> + }
>
>
>
> newLen = ReadMatchDistances(p, &numPairs);
>
>
>
> @@ -1294,15 +1380,24 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> p->numPairs = numPairs;
>
> p->longestMatchLen = newLen;
>
> - return Backward(p, cur);
>
> + break;
>
> }
>
>
>
> curOpt = &p->opt[cur];
>
> +
>
> + position++;
>
> +
>
> + // we need that check here, if skip_items in p->opt are possible
>
> + /*
>
> + if (curOpt->price >= kInfinityPrice)
>
> + continue;
>
> + */
>
> +
>
> prev = cur - curOpt->len;
>
> -
>
> +
>
> if (curOpt->len == 1)
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (IsShortRep(curOpt))
>
> state = kShortRepNextStates[state];
>
> else
>
> @@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> if (curOpt->extra)
>
> {
>
> - prev -= curOpt->extra;
>
> + prev -= (unsigned)curOpt->extra;
>
> state = kState_RepAfterLit;
>
> if (curOpt->extra == 1)
>
> - state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit :
> kState_MatchAfterLit;
>
> + state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit :
> kState_MatchAfterLit);
>
> }
>
> else
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (dist < LZMA_NUM_REPS)
>
> state = kRepNextStates[state];
>
> else
>
> @@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
>
>
> - position++;
>
> posState = (position & p->pbMask);
>
>
>
> /*
>
> @@ -1391,13 +1485,25 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> < MATCH [ : LIT : REP_0 ]
>
> */
>
>
>
> - curPrice = curOpt->price;
>
> - litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
>
> + {
>
> + UInt32 curPrice = curOpt->price;
>
> + unsigned prob = p->isMatch[state][posState];
>
> + matchPrice = curPrice + GET_PRICE_1(prob);
>
> + litPrice = curPrice + GET_PRICE_0(prob);
>
> + }
>
>
>
> nextOpt = &p->opt[(size_t)cur + 1];
>
> nextIsLit = False;
>
>
>
> - // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
>
> + // here we can allow skip_items in p->opt, if we don't check
> (nextOpt->price < kInfinityPrice)
>
> + // 18.new.06
>
> + if ((nextOpt->price < kInfinityPrice
>
> + // && !IsLitState(state)
>
> + && matchByte == curByte)
>
> + || litPrice > nextOpt->price
>
> + )
>
> + litPrice = 0;
>
> + else
>
> {
>
> const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
>
> litPrice += (!IsLitState(state) ?
>
> @@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
>
>
>
> + numAvailFull = p->numAvail;
>
> + {
>
> + unsigned temp = kNumOpts - 1 - cur;
>
> + if (numAvailFull > temp)
>
> + numAvailFull = (UInt32)temp;
>
> + }
>
> +
>
> + // 18.06
>
> // ---------- SHORT_REP ----------
>
> - // if (IsLitState(state)) // 18.new
>
> + if (IsLitState(state)) // 18.new
>
> if (matchByte == curByte)
>
> - // if (repMatchPrice < nextOpt->price) // 18.new
>
> - if (nextOpt->len < 2
>
> + if (repMatchPrice < nextOpt->price) // 18.new
>
> + // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
>
> + if (
>
> + // nextOpt->price >= kInfinityPrice ||
>
> + nextOpt->len < 2 // we can check nextOpt->len, if skip items
> are not allowed in p->opt
>
> || (nextOpt->dist != 0
>
> - && nextOpt->extra <= 1 // 17.old
>
> - ))
>
> + // && nextOpt->extra <= 1 // 17.old
>
> + )
>
> + )
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state,
> posState);
>
> - if (shortRepPrice <= nextOpt->price) // 17.old
>
> - // if (shortRepPrice < nextOpt->price) // 18.new
>
> + // if (shortRepPrice <= nextOpt->price) // 17.old
>
> + if (shortRepPrice < nextOpt->price) // 18.new
>
> {
>
> nextOpt->price = shortRepPrice;
>
> nextOpt->len = 1;
>
> @@ -1436,13 +1553,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - numAvailFull = p->numAvail;
>
> - {
>
> - UInt32 temp = kNumOpts - 1 - cur;
>
> - if (numAvailFull > temp)
>
> - numAvailFull = temp;
>
> - }
>
> -
>
> if (numAvailFull < 2)
>
> continue;
>
> numAvail = (numAvailFull <= p->numFastBytes ? numAvailFull :
> p->numFastBytes);
>
> @@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
>
>
> // ---------- LIT : REP_0 ----------
>
>
>
> - if (
>
> - // litPrice != 0 && // 18.new
>
> - !nextIsLit
>
> + if (!nextIsLit
>
> + && litPrice != 0 // 18.new
>
> && matchByte != curByte
>
> && numAvailFull > 2)
>
> {
>
> @@ -1465,8 +1574,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> for (len = 3; len < limit && data[len] == data2[len]; len++)
>
> - {
>
> - }
>
> + {}
>
>
>
> {
>
> unsigned state2 = kLiteralNextStates[state];
>
> @@ -1474,8 +1582,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
>
> {
>
> unsigned offset = cur + len;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
>
>
> // do
>
> {
>
> @@ -1483,14 +1592,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> COptimal *opt;
>
> len--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = 0;
>
> opt->extra = 1;
>
> }
>
> @@ -1502,6 +1611,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> }
>
>
>
> startLen = 2; /* speed optimization */
>
> +
>
> {
>
> // ---------- REP ----------
>
> unsigned repIndex = 0; // 17.old
>
> @@ -1514,24 +1624,28 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
>
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
>
>
> // if (len < startLen) continue; // 18.new: speed optimization
>
>
>
> - while (last < cur + len)
>
> - p->opt[++last].price = kInfinityPrice;
>
> + {
>
> + unsigned offset = cur + len;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
> {
>
> unsigned len2 = len;
>
> price = repMatchPrice + GetPrice_PureRep(p, repIndex, state,
> posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)len2 - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState, len2);
>
> COptimal *opt = &p->opt[cur + len2];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> - opt->dist = repIndex;
>
> + opt->len = (UInt32)len2;
>
> + opt->dist = (UInt32)repIndex;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1551,15 +1665,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> -
>
> - len2 -= len;
>
> - if (len2 >= 3)
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> {
>
> unsigned state2 = kRepNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> - price +=
>
> - p->repLenEnc.prices[posState][(size_t)len - 2]
>
> + price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
>
> + GET_PRICE_0(p->isMatch[state2][posState2])
>
> + LitEnc_Matched_GetPrice(LIT_PROBS(position + len,
> data[(size_t)len - 1]),
>
> data[len], data2[len], p->ProbPrices);
>
> @@ -1570,31 +1683,40 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
>
>
> price += GetPrice_Rep_0(p, state2, posState2);
>
> +
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
> +
>
> + len2 -= len;
>
> + // if (len2 >= 3)
>
> + {
>
> {
>
> unsigned offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> - unsigned price2;
>
> + UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> - opt->dist = repIndex;
>
> + opt->dist = (UInt32)repIndex;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> }
>
> + }
>
> }
>
> }
>
> }
>
> @@ -1606,17 +1728,23 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> newLen = numAvail;
>
> for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
>
> - matches[numPairs] = newLen;
>
> + matches[numPairs] = (UInt32)newLen;
>
> numPairs += 2;
>
> }
>
>
>
> + // startLen = 2; /* speed optimization */
>
> +
>
> if (newLen >= startLen)
>
> {
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[state]);
>
> UInt32 dist;
>
> unsigned offs, posSlot, len;
>
> - while (last < cur + newLen)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + {
>
> + unsigned offset = cur + newLen;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
>
>
> offs = 0;
>
> while (startLen > matches[offs])
>
> @@ -1628,27 +1756,29 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (len = /*2*/ startLen; ; len++)
>
> {
>
> - UInt32 price = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc,
> posState, len);
>
> {
>
> COptimal *opt;
>
> - unsigned lenToPosState = len - 2; lenToPosState =
> GetLenToPosState2(lenToPosState);
>
> + unsigned lenNorm = len - 2;
>
> + lenNorm = GetLenToPosState2(lenNorm);
>
> if (dist < kNumFullDistances)
>
> - price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenNorm][dist &
> (kNumFullDistances - 1)];
>
> else
>
> - price += p->posSlotPrices[lenToPosState][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenNorm][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
>
>
> opt = &p->opt[cur + len];
>
> if (price < opt->price)
>
> {
>
> opt->price = price;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> }
>
>
>
> - if (/*_maxMode && */ len == matches[offs])
>
> + if (len == matches[offs])
>
> {
>
> + // if (p->_maxMode) {
>
> // MATCH : LIT : REP_0
>
>
>
> const Byte *data2 = data - dist - 1;
>
> @@ -1657,11 +1787,17 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> + {
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
>
>
> len2 -= len;
>
>
>
> - if (len2 >= 3)
>
> + // if (len2 >= 3)
>
> {
>
> unsigned state2 = kMatchNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> @@ -1677,27 +1813,30 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price += GetPrice_Rep_0(p, state2, posState2);
>
>
>
> offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> +
>
> + }
>
>
>
> offs += 2;
>
> if (offs == numPairs)
>
> @@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
> }
>
> +
>
> + do
>
> + p->opt[last].price = kInfinityPrice;
>
> + while (--last);
>
> +
>
> + return Backward(p, cur);
>
> }
>
>
>
>
>
> @@ -1735,6 +1880,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> p->backRes = MARK_LIT;
>
> if (numAvail < 2)
>
> return 1;
>
> + // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
>
> if (numAvail > LZMA_MATCH_LEN_MAX)
>
> numAvail = LZMA_MATCH_LEN_MAX;
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> @@ -1746,10 +1892,11 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> const Byte *data2 = data - p->reps[i];
>
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> if (len >= p->numFastBytes)
>
> {
>
> - p->backRes = i;
>
> + p->backRes = (UInt32)i;
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> }
>
> @@ -1793,7 +1940,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> || (repLen + 2 >= mainLen && mainDist >= (1 << 9))
>
> || (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
>
> {
>
> - p->backRes = repIndex;
>
> + p->backRes = (UInt32)repIndex;
>
> MOVE_POS(p, repLen - 1)
>
> return repLen;
>
> }
>
> @@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p,
> UInt32 nowPos)
> }
>
>
>
>
>
> -
>
> -static void FillAlignPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
>
> {
>
> unsigned i;
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> const CLzmaProb *probs = p->posAlignEncoder;
>
> - p->alignPriceCount = 0;
>
> + // p->alignPriceCount = 0;
>
> for (i = 0; i < kAlignTableSize / 2; i++)
>
> {
>
> UInt32 price = 0;
>
> - unsigned symbol = i;
>
> + unsigned sym = i;
>
> unsigned m = 1;
>
> unsigned bit;
>
> UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> prob = probs[m];
>
> p->alignPrices[i ] = price + GET_PRICEa_0(prob);
>
> p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
>
> @@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
> }
>
>
>
>
>
> -static void FillDistancesPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
>
> {
>
> + // int y; for (y = 0; y < 100; y++) {
>
> +
>
> UInt32 tempPrices[kNumFullDistances];
>
> - unsigned i, lenToPosState;
>
> + unsigned i, lps;
>
>
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> p->matchPriceCount = 0;
>
>
>
> - for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
>
> + for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
>
> {
>
> unsigned posSlot = GetPosSlot1(i);
>
> - unsigned footerBits = ((posSlot >> 1) - 1);
>
> + unsigned footerBits = (posSlot >> 1) - 1;
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> + const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
>
> // tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base,
> footerBits, i - base, p->ProbPrices);
>
> -
>
> - const CLzmaProb *probs = p->posEncoders + base;
>
> UInt32 price = 0;
>
> unsigned m = 1;
>
> - unsigned symbol = i - base;
>
> + unsigned sym = i;
>
> + unsigned offset = (unsigned)1 << footerBits;
>
> + base += i;
>
> +
>
> + if (footerBits)
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> price += GET_PRICEa(probs[m], bit);
>
> m = (m << 1) + bit;
>
> }
>
> while (--footerBits);
>
> - tempPrices[i] = price;
>
> +
>
> + {
>
> + unsigned prob = probs[m];
>
> + tempPrices[base ] = price + GET_PRICEa_0(prob);
>
> + tempPrices[base + offset] = price + GET_PRICEa_1(prob);
>
> + }
>
> }
>
>
>
> - for (lenToPosState = 0; lenToPosState < kNumLenToPosStates;
> lenToPosState++)
>
> + for (lps = 0; lps < kNumLenToPosStates; lps++)
>
> {
>
> - unsigned posSlot;
>
> - const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
>
> - UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
>
> - unsigned distTableSize = p->distTableSize;
>
> - const CLzmaProb *probs = encoder;
>
> - for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
>
> + unsigned slot;
>
> + unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
>
> + UInt32 *posSlotPrices = p->posSlotPrices[lps];
>
> + const CLzmaProb *probs = p->posSlotEncoder[lps];
>
> +
>
> + for (slot = 0; slot < distTableSize2; slot++)
>
> {
>
> - // posSlotPrices[posSlot] = RcTree_GetPrice(encoder,
> kNumPosSlotBits, posSlot, p->ProbPrices);
>
> - UInt32 price = 0;
>
> + // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits,
> slot, p->ProbPrices);
>
> + UInt32 price;
>
> unsigned bit;
>
> - unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
>
> - UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
>
> - posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
>
> - posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
>
> + unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
>
> + unsigned prob;
>
> + bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
>
> + posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
>
> + posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
>
> }
>
> - for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
>
> - posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> -
>
> +
>
> {
>
> - UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
>
> + UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> + for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
>
> {
>
> - distancesPrices[0] = posSlotPrices[0];
>
> - distancesPrices[1] = posSlotPrices[1];
>
> - distancesPrices[2] = posSlotPrices[2];
>
> - distancesPrices[3] = posSlotPrices[3];
>
> + posSlotPrices[(size_t)slot * 2 ] += delta;
>
> + posSlotPrices[(size_t)slot * 2 + 1] += delta;
>
> + delta += ((UInt32)1 << kNumBitPriceShiftBits);
>
> }
>
> + }
>
> +
>
> + {
>
> + UInt32 *dp = p->distancesPrices[lps];
>
> +
>
> + dp[0] = posSlotPrices[0];
>
> + dp[1] = posSlotPrices[1];
>
> + dp[2] = posSlotPrices[2];
>
> + dp[3] = posSlotPrices[3];
>
> +
>
> for (i = 4; i < kNumFullDistances; i += 2)
>
> {
>
> UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
>
> - distancesPrices[i ] = slotPrice + tempPrices[i];
>
> - distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
>
> + dp[i ] = slotPrice + tempPrices[i];
>
> + dp[i + 1] = slotPrice + tempPrices[i + 1];
>
> }
>
> }
>
> }
>
> + // }
>
> }
>
>
>
>
>
> @@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> if (len != 1)
>
> {
>
> LenEnc_Encode(&p->repLenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->repLenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->repLenEnc, posState,
> &p->repLenProbs, p->ProbPrices);
>
> -
>
> + --p->repLenEncCounter;
>
> p->state = kRepNextStates[p->state];
>
> }
>
> }
>
> @@ -2260,9 +2422,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> p->state = kMatchNextStates[p->state];
>
>
>
> LenEnc_Encode(&p->lenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->lenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->lenEnc, posState,
> &p->lenProbs, p->ProbPrices);
>
> + // --p->lenEnc.counter;
>
>
>
> dist -= LZMA_NUM_REPS;
>
> p->reps[3] = p->reps[2];
>
> @@ -2274,17 +2434,17 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> GetPosSlot(dist, posSlot);
>
> // RcTree_Encode_PosSlot(&p->rc,
> p->posSlotEncoder[GetLenToPosState(len)], posSlot);
>
> {
>
> - UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
>
> + UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
>
> range = p->rc.range;
>
> probs = p->posSlotEncoder[GetLenToPosState(len)];
>
> do
>
> {
>
> - CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
>
> - UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
>
> - symbol <<= 1;
>
> + CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
>
> + UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
>
> + sym <<= 1;
>
> RC_BIT(&p->rc, prob, bit);
>
> }
>
> - while (symbol < (1 << kNumPosSlotBits * 2));
>
> + while (sym < (1 << kNumPosSlotBits * 2));
>
> p->rc.range = range;
>
> }
>
>
>
> @@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> if (dist < kNumFullDistances)
>
> {
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> - RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, dist - base);
>
> + RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, (unsigned)(dist /* - base */));
>
> }
>
> else
>
> {
>
> @@ -2331,14 +2491,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> bit = dist & 1; dist >>= 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit); m = (m << 1) + bit;
>
> bit = dist & 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit);
>
> p->rc.range = range;
>
> - p->alignPriceCount++;
>
> + // p->alignPriceCount++;
>
> }
>
> }
>
> }
>
> }
>
> }
>
>
>
> - nowPos32 += len;
>
> + nowPos32 += (UInt32)len;
>
> p->additionalOffset -= len;
>
>
>
> if (p->additionalOffset == 0)
>
> @@ -2347,10 +2507,27 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
>
>
> if (!p->fastMode)
>
> {
>
> - if (p->matchPriceCount >= (1 << 7))
>
> + /*
>
> + if (p->alignPriceCount >= 16) // kAlignTableSize
>
> + FillAlignPrices(p);
>
> + if (p->matchPriceCount >= 128)
>
> FillDistancesPrices(p);
>
> - if (p->alignPriceCount >= kAlignTableSize)
>
> + if (p->lenEnc.counter <= 0)
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + */
>
> + if (p->matchPriceCount >= 64)
>
> + {
>
> FillAlignPrices(p);
>
> + // { int y; for (y = 0; y < 100; y++) {
>
> + FillDistancesPrices(p);
>
> + // }}
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + }
>
> + if (p->repLenEncCounter <= 0)
>
> + {
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> + LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb,
> &p->repLenProbs, p->ProbPrices);
>
> + }
>
> }
>
>
>
> if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
>
> @@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
>
>
> p->optEnd = 0;
>
> p->optCur = 0;
>
> +
>
> + {
>
> + for (i = 0; i < kNumOpts; i++)
>
> + p->opt[i].price = kInfinityPrice;
>
> + }
>
> +
>
> p->additionalOffset = 0;
>
>
>
> p->pbMask = (1 << p->pb) - 1;
>
> p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
>
> }
>
>
>
> +
>
> void LzmaEnc_InitPrices(CLzmaEnc *p)
>
> {
>
> if (!p->fastMode)
>
> @@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
> p->lenEnc.tableSize =
>
> p->repLenEnc.tableSize =
>
> p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
>
> +
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> +
>
> LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs,
> p->ProbPrices);
>
> LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs,
> p->ProbPrices);
>
> }
>
> @@ -2583,7 +2770,7 @@ typedef struct
> ISeqOutStream vt;
>
> Byte *data;
>
> SizeT rem;
>
> - Bool overflow;
>
> + BoolInt overflow;
>
> } CLzmaEnc_SeqOutStreamBuf;
>
>
>
> static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void
> *data, size_t size)
>
> @@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle
> pp)
> }
>
>
>
>
>
> -SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
>
> +SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
>
> Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32
> *unpackSize)
>
> {
>
> CLzmaEnc *p = (CLzmaEnc *)pp;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> index d7426d38a4..04f9d98e7d 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> @@ -1,6 +1,28 @@
> HISTORY of the LZMA SDK
>
> -----------------------
>
>
>
> +19.00 2019-02-21
>
> +-------------------------
>
> +- Encryption strength for 7z archives was increased:
>
> + the size of random initialization vector was increased from 64-bit to
> 128-bit,
>
> + and the pseudo-random number generator was improved.
>
> +- The bug in 7zIn.c code was fixed.
>
> +
>
> +
>
> +18.06 2018-12-30
>
> +-------------------------
>
> +- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
>
> + and there are minor changes in compression ratio.
>
> +- Some bugs were fixed.
>
> +- The bug in 7-Zip 18.02-18.05 was fixed:
>
> + There was memory leak in multithreading xz decoder -
> XzDecMt_Decode(),
>
> + if xz stream contains only one block.
>
> +- The changes for MSVS compiler makefiles:
>
> + - the makefiles now use "PLATFORM" macroname with values (x64, x86,
> arm64)
>
> + instead of "CPU" macroname with values (AMD64, ARM64).
>
> + - the makefiles by default now use static version of the run-time
library.
>
> +
>
> +
>
> 18.05 2018-04-30
>
> -------------------------
>
> - The speed for LZMA/LZMA2 compressing was increased
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> index a3deb20943..e17cd4fe1a 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> @@ -1,4 +1,4 @@
> -LZMA SDK 18.05
>
> +LZMA SDK 19.00
>
> --------------
>
>
>
> LZMA SDK provides the documentation, samples, header files,
>
> --
> 2.27.0.windows.1
[-- Attachment #2: Type: text/html, Size: 484232 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
2020-12-21 1:10 ` 回复: " gaoliming
@ 2020-12-21 7:26 ` weix.c.liu
0 siblings, 0 replies; 7+ messages in thread
From: weix.c.liu @ 2020-12-21 7:26 UTC (permalink / raw)
To: gaoliming, devel@edk2.groups.io; +Cc: 'Laszlo Ersek'
[-- Attachment #1: Type: text/plain, Size: 112182 bytes --]
Yes. Liming, I have also checked boot functionality (boot to shell) and image size. All pass and no size change.
For windows: Base e6ae24e1d676bb2bdc0fc715b49b04908f41fc10
For linux: Base e6ae24e1d676bb2bdc0fc715b49b04908f41fc10
Thanks & BRs,
LIUWEI
From: gaoliming <gaoliming@byosoft.com.cn>
Sent: 2020年12月21日 9:11
To: Liu, WeiX C <weix.c.liu@intel.com>; devel@edk2.groups.io
Cc: 'Laszlo Ersek' <lersek@redhat.com>
Subject: 回复: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
Do you verify the boot functionality (boot to shell)? Do you collect the image size (old LZMA version and new LZMA version)?
Thanks
Liming
发件人: Liu, WeiX C <weix.c.liu@intel.com<mailto:weix.c.liu@intel.com>>
发送时间: 2020年12月20日 21:34
收件人: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
抄送: 'Laszlo Ersek' <lersek@redhat.com<mailto:lersek@redhat.com>>
主题: RE: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
Hi liming,
I listed unittest result as below with Ovmf build, cause I have no apple machine so only list VS、GCC and clang tool chain .
VS2019
GCC5
CLANGPDB
IA32
Pass
Pass
Pass
X64
Pass
Pass
Pass
IA32x64
Pass
Pass
Pass
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
Sent: 2020年12月17日 9:30
To: Liu, WeiX C <weix.c.liu@intel.com<mailto:weix.c.liu@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: 'Laszlo Ersek' <lersek@redhat.com<mailto:lersek@redhat.com>>
Subject: 回复: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version
Wei:
Can you list the unit test for this patch? Which tool chain and arch have
been verified?
Because LZMA is well used, I request to verify it on VS, GCC, XCODE and
CLANG tool chain for the different ARCHs.
Thanks
Liming
> -----邮件原件-----
> 发件人: Wei Liu <weix.c.liu@intel.com<mailto:weix.c.liu@intel.com>>
> 发送时间: 2020年12月15日 10:07
> 收件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> 抄送: Wei Liu <weix.c.liu@intel.com<mailto:weix.c.liu@intel.com>>; Liming Gao
> <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>
> 主题: [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00
> version
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3101
> New formal release in https://www.7-zip.org/sdk.html is 19.00.
>
> Cc: Liming Gao <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
>
> Cc: Laszlo Ersek <lersek@redhat.com<mailto:lersek@redhat.com>>
> Signed-off-by: Wei Liu <weix.c.liu@intel.com<mailto:weix.c.liu@intel.com>>
> ---
> BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt | 4
> ++--
> BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 5
> +++--
> BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 8
> ++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 9
> +++++----
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 164
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++++++++++++++++++++++++++++++++++++++--------------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 117
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++------------------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14
> +++++++-------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 32
> ++++++++++++++++----------------
> BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 733
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
++++++++++++++++++++++++++--------------------------------------------------
----
>
----------------------------------------------------------------------------
-----------------
>
----------------------------------------------------------------------------
-----------------
> ---------------------------------
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 22
> ++++++++++++++++++++++
> BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 2 +-
> 12 files changed, 710 insertions(+), 408 deletions(-)
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> index 7cf967a774..b2338e2598 100644
> --- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> +++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt
> @@ -1,3 +1,3 @@
> -LzmaCompress is based on the LZMA SDK 18.05. LZMA SDK 18.05
>
> -was placed in the public domain on 2018-04-30. It was
>
> +LzmaCompress is based on the LZMA SDK 19.00. LZMA SDK 19.00
>
> +was placed in the public domain on 2019-02-21. It was
>
> released on the http://www.7-zip.org/sdk.html website.
>
> diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> index bebdb9aa84..bf4706ee23 100644
> --- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> +++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c
> @@ -1,9 +1,9 @@
> /** @file
>
> LZMA Compress/Decompress tool (LzmaCompress)
>
>
>
> - Based on LZMA SDK 18.05:
>
> + Based on LZMA SDK 19.00:
>
> LzmaUtil.c -- Test application for LZMA compression
>
> - 2018-04-30 : Igor Pavlov : Public domain
>
> + 2019-02-21 : Igor Pavlov : Public domain
>
>
>
> Copyright (c) 2006 - 2019, Intel Corporation. All rights reserved.<BR>
>
> SPDX-License-Identifier: BSD-2-Clause-Patent
>
> @@ -39,7 +39,7 @@ const char *kCantAllocateMessage = "Can not allocate
> memory";
> const char *kDataErrorMessage = "Data error";
>
> const char *kInvalidParamValMessage = "Invalid parameter value";
>
>
>
> -static Bool mQuietMode = False;
>
> +static BoolInt mQuietMode = False;
>
> static CONVERTER_TYPE mConType = NoConverter;
>
>
>
> UINT64 mDictionarySize = 28;
>
> @@ -244,7 +244,7 @@ int main2(int numArgs, const char *args[], char *rs)
> CFileOutStream outStream;
>
> int res;
>
> int encodeMode = 0;
>
> - Bool modeWasSet = False;
>
> + BoolInt modeWasSet = False;
>
> const char *inputFile = NULL;
>
> const char *outputFile = "file.tmp";
>
> int param;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> index 4977cdaa66..593f5aa259 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h
> @@ -1,5 +1,5 @@
> /* 7zTypes.h -- Basic types
>
> -2017-07-17 : Igor Pavlov : Public domain */
>
> +2018-08-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __7Z_TYPES_H
>
> #define __7Z_TYPES_H
>
> @@ -103,7 +103,8 @@ typedef UInt32 SizeT;
> typedef size_t SizeT;
>
> #endif
>
>
>
> -typedef int Bool;
>
> +typedef int BoolInt;
>
> +/* typedef BoolInt Bool; */
>
> #define True 1
>
> #define False 0
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> index ed3aa94270..0074c64be9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h
> @@ -1,7 +1,7 @@
> -#define MY_VER_MAJOR 18
>
> -#define MY_VER_MINOR 05
>
> +#define MY_VER_MAJOR 19
>
> +#define MY_VER_MINOR 00
>
> #define MY_VER_BUILD 0
>
> -#define MY_VERSION_NUMBERS "18.05"
>
> +#define MY_VERSION_NUMBERS "19.00"
>
> #define MY_VERSION MY_VERSION_NUMBERS
>
>
>
> #ifdef MY_CPU_NAME
>
> @@ -10,7 +10,7 @@
> #define MY_VERSION_CPU MY_VERSION
>
> #endif
>
>
>
> -#define MY_DATE "2018-04-30"
>
> +#define MY_DATE "2019-02-21"
>
> #undef MY_COPYRIGHT
>
> #undef MY_VERSION_COPYRIGHT_DATE
>
> #define MY_AUTHOR_NAME "Igor Pavlov"
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> index 7fb27282c7..5f74c1c0cb 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h
> @@ -1,5 +1,5 @@
> /* CpuArch.h -- CPU specific code
>
> -2017-09-04 : Igor Pavlov : Public domain */
>
> +2018-02-18 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __CPU_ARCH_H
>
> #define __CPU_ARCH_H
>
> @@ -318,15 +318,16 @@ enum
>
>
> void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32
*d);
>
>
>
> -Bool x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> +BoolInt x86cpuid_CheckAndRead(Cx86cpuid *p);
>
> int x86cpuid_GetFirm(const Cx86cpuid *p);
>
>
>
> #define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) &
0xF))
>
> #define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) &
> 0xF))
>
> #define x86cpuid_GetStepping(ver) (ver & 0xF)
>
>
>
> -Bool CPU_Is_InOrder();
>
> -Bool CPU_Is_Aes_Supported();
>
> +BoolInt CPU_Is_InOrder();
>
> +BoolInt CPU_Is_Aes_Supported();
>
> +BoolInt CPU_IsSupported_PageGB();
>
>
>
> #endif
>
>
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> index 6ea82a9b53..4eefc17dd2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c
> @@ -1,5 +1,5 @@
> /* LzFind.c -- Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-07-08 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -138,7 +138,7 @@ static void
> MatchFinder_SetDefaultSettings(CMatchFinder *p)
>
>
> void MatchFinder_Construct(CMatchFinder *p)
>
> {
>
> - UInt32 i;
>
> + unsigned i;
>
> p->bufferBase = NULL;
>
> p->directInput = 0;
>
> p->hash = NULL;
>
> @@ -147,7 +147,7 @@ void MatchFinder_Construct(CMatchFinder *p)
>
>
> for (i = 0; i < 256; i++)
>
> {
>
> - UInt32 r = i;
>
> + UInt32 r = (UInt32)i;
>
> unsigned j;
>
> for (j = 0; j < 8; j++)
>
> r = (r >> 1) ^ (kCrcPoly & ((UInt32)0 - (r & 1)));
>
> @@ -368,6 +368,8 @@ static void MatchFinder_Normalize(CMatchFinder *p)
> MatchFinder_ReduceOffsets(p, subValue);
>
> }
>
>
>
> +
>
> +MY_NO_INLINE
>
> static void MatchFinder_CheckLimits(CMatchFinder *p)
>
> {
>
> if (p->pos == kMaxValForNormalize)
>
> @@ -379,10 +381,16 @@ static void
> MatchFinder_CheckLimits(CMatchFinder *p)
> MatchFinder_SetLimits(p);
>
> }
>
>
>
> -static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> +
>
> +/*
>
> + (lenLimit > maxLen)
>
> +*/
>
> +MY_FORCE_INLINE
>
> +static UInt32 * Hc_GetMatchesSpec(unsigned lenLimit, UInt32 curMatch,
> UInt32 pos, const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> - UInt32 *distances, UInt32 maxLen)
>
> + UInt32 *distances, unsigned maxLen)
>
> {
>
> + /*
>
> son[_cyclicBufferPos] = curMatch;
>
> for (;;)
>
> {
>
> @@ -400,7 +408,8 @@ static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos,
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> return distances;
>
> @@ -408,15 +417,58 @@ static UInt32 * Hc_GetMatchesSpec(UInt32
> lenLimit, UInt32 curMatch, UInt32 pos,
> }
>
> }
>
> }
>
> + */
>
> +
>
> + const Byte *lim = cur + lenLimit;
>
> + son[_cyclicBufferPos] = curMatch;
>
> + do
>
> + {
>
> + UInt32 delta = pos - curMatch;
>
> + if (delta >= _cyclicBufferSize)
>
> + break;
>
> + {
>
> + ptrdiff_t diff;
>
> + curMatch = son[_cyclicBufferPos - delta + ((delta >
_cyclicBufferPos) ?
> _cyclicBufferSize : 0)];
>
> + diff = (ptrdiff_t)0 - delta;
>
> + if (cur[maxLen] == cur[maxLen + diff])
>
> + {
>
> + const Byte *c = cur;
>
> + while (*c == c[diff])
>
> + {
>
> + if (++c == lim)
>
> + {
>
> + distances[0] = (UInt32)(lim - cur);
>
> + distances[1] = delta - 1;
>
> + return distances + 2;
>
> + }
>
> + }
>
> + {
>
> + unsigned len = (unsigned)(c - cur);
>
> + if (maxLen < len)
>
> + {
>
> + maxLen = len;
>
> + distances[0] = (UInt32)len;
>
> + distances[1] = delta - 1;
>
> + distances += 2;
>
> + }
>
> + }
>
> + }
>
> + }
>
> + }
>
> + while (--cutValue);
>
> +
>
> + return distances;
>
> }
>
>
>
> +
>
> +MY_FORCE_INLINE
>
> UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
>
> UInt32 *distances, UInt32 maxLen)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -426,9 +478,10 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> return distances;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = pair[0];
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -437,11 +490,12 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const Byt
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = (UInt32)len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> + *ptr1 = pair0;
>
> *ptr0 = pair[1];
>
> return distances;
>
> }
>
> @@ -468,9 +522,9 @@ UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const Byt
> static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
>
> {
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> for (;;)
>
> {
>
> UInt32 delta = pos - curMatch;
>
> @@ -480,9 +534,9 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32
> curMatch, UInt32 pos, const
> return;
>
> }
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> if (pb[len] == cur[len])
>
> {
>
> while (++len != lenLimit)
>
> @@ -520,13 +574,13 @@ static void SkipMatchesSpec(UInt32 lenLimit,
> UInt32 curMatch, UInt32 pos, const
> p->buffer++; \
>
> if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
>
>
>
> -#define MOVE_POS_RET MOVE_POS return offset;
>
> +#define MOVE_POS_RET MOVE_POS return (UInt32)offset;
>
>
>
> static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
>
>
>
> #define GET_MATCHES_HEADER2(minLen, ret_op) \
>
> - UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> - lenLimit = p->lenLimit; { if (lenLimit < minLen) {
MatchFinder_MovePos(p);
> ret_op; }} \
>
> + unsigned lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \
>
> + lenLimit = (unsigned)p->lenLimit; { if (lenLimit < minLen)
> { MatchFinder_MovePos(p); ret_op; }} \
>
> cur = p->buffer;
>
>
>
> #define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen,
> return 0)
>
> @@ -535,22 +589,22 @@ static void MatchFinder_MovePos(CMatchFinder
> *p) { MOVE_POS; }
> #define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos,
> p->cyclicBufferSize, p->cutValue
>
>
>
> #define GET_MATCHES_FOOTER(offset, maxLen) \
>
> - offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
>
> - distances + offset, maxLen) - distances); MOVE_POS_RET;
>
> + offset = (unsigned)(GetMatchesSpec1((UInt32)lenLimit, curMatch,
> MF_PARAMS(p), \
>
> + distances + offset, (UInt32)maxLen) - distances); MOVE_POS_RET;
>
>
>
> #define SKIP_FOOTER \
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
> MOVE_POS;
>
>
>
> #define UPDATE_maxLen { \
>
> ptrdiff_t diff = (ptrdiff_t)0 - d2; \
>
> const Byte *c = cur + maxLen; \
>
> const Byte *lim = cur + lenLimit; \
>
> for (; c != lim; c++) if (*(c + diff) != *c) break; \
>
> - maxLen = (UInt32)(c - cur); }
>
> + maxLen = (unsigned)(c - cur); }
>
>
>
> static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(2)
>
> HASH2_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -561,7 +615,7 @@ static UInt32
> Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> @@ -572,7 +626,8 @@ UInt32
> Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, d2, maxLen, offset, pos;
>
> + UInt32 h2, d2, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(3)
>
>
>
> @@ -594,12 +649,12 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> UPDATE_maxLen
>
> - distances[0] = maxLen;
>
> + distances[0] = (UInt32)maxLen;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -609,7 +664,8 @@ static UInt32
> Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -618,12 +674,12 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
>
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -632,7 +688,8 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -648,10 +705,10 @@ static UInt32
> Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> - SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
>
> + SkipMatchesSpec((UInt32)lenLimit, curMatch, MF_PARAMS(p));
>
> MOVE_POS_RET;
>
> }
>
> }
>
> @@ -674,13 +731,13 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -741,7 +798,8 @@ static UInt32
> Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 h2, h3, d2, d3, maxLen, offset, pos;
>
> + UInt32 h2, h3, d2, d3, pos;
>
> + unsigned maxLen, offset;
>
> UInt32 *hash;
>
> GET_MATCHES_HEADER(4)
>
>
>
> @@ -750,12 +808,11 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> -
>
> curMatch = (hash + kFix4HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[hv] = pos;
>
>
>
> @@ -764,7 +821,8 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> if (d2 < p->cyclicBufferSize && *(cur - d2) == *cur)
>
> {
>
> - distances[0] = maxLen = 2;
>
> + maxLen = 2;
>
> + distances[0] = 2;
>
> distances[1] = d2 - 1;
>
> offset = 2;
>
> }
>
> @@ -780,7 +838,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (offset != 0)
>
> {
>
> UPDATE_maxLen
>
> - distances[(size_t)offset - 2] = maxLen;
>
> + distances[(size_t)offset - 2] = (UInt32)maxLen;
>
> if (maxLen == lenLimit)
>
> {
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -791,7 +849,7 @@ static UInt32
> Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> if (maxLen < 3)
>
> maxLen = 3;
>
>
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances + offset, maxLen) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -808,13 +866,13 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
> hash = p->hash;
>
> pos = p->pos;
>
>
>
> - d2 = pos - hash[ h2];
>
> + d2 = pos - hash [h2];
>
> d3 = pos - (hash + kFix3HashSize)[h3];
>
> d4 = pos - (hash + kFix4HashSize)[h4];
>
>
>
> curMatch = (hash + kFix5HashSize)[hv];
>
>
>
> - hash[ h2] = pos;
>
> + hash [h2] = pos;
>
> (hash + kFix3HashSize)[h3] = pos;
>
> (hash + kFix4HashSize)[h4] = pos;
>
> (hash + kFix5HashSize)[hv] = pos;
>
> @@ -877,12 +935,12 @@ static UInt32
> Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
>
>
> UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32
> *distances)
>
> {
>
> - UInt32 offset;
>
> + unsigned offset;
>
> GET_MATCHES_HEADER(3)
>
> HASH_ZIP_CALC;
>
> curMatch = p->hash[hv];
>
> p->hash[hv] = p->pos;
>
> - offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
>
> + offset = (unsigned)(Hc_GetMatchesSpec(lenLimit, curMatch,
> MF_PARAMS(p),
>
> distances, 2) - (distances));
>
> MOVE_POS_RET
>
> }
>
> @@ -940,7 +998,7 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> SKIP_FOOTER
>
> @@ -959,7 +1017,7 @@ static void Bt5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> @@ -979,7 +1037,7 @@ static void Hc4_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH4_CALC;
>
> hash = p->hash;
>
> curMatch = (hash + kFix4HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[hv] = p->pos;
>
> p->son[p->cyclicBufferPos] = curMatch;
>
> @@ -999,7 +1057,7 @@ static void Hc5_MatchFinder_Skip(CMatchFinder *p,
> UInt32 num)
> HASH5_CALC;
>
> hash = p->hash;
>
> curMatch = hash + kFix5HashSize)[hv];
>
> - hash[ h2] =
>
> + hash [h2] =
>
> (hash + kFix3HashSize)[h3] =
>
> (hash + kFix4HashSize)[h4] =
>
> (hash + kFix5HashSize)[hv] = p->pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> index 2563824fcd..df32146f92 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c
> @@ -1,5 +1,5 @@
> /* LzFindMt.c -- multithreaded Match finder for LZ algorithms
>
> -2017-06-10 : Igor Pavlov : Public domain */
>
> +2018-12-29 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -232,38 +232,57 @@ static void
> MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p)
>
>
> #define kEmptyHashValue 0
>
>
>
> -/* #define MFMT_GM_INLINE */
>
> +#define MFMT_GM_INLINE
>
>
>
> #ifdef MFMT_GM_INLINE
>
>
>
> -#define NO_INLINE MY_FAST_CALL
>
> +/*
>
> + we use size_t for _cyclicBufferPos instead of UInt32
>
> + to eliminate "movsx" BUG in old MSVC x64 compiler.
>
> +*/
>
>
>
> -static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos,
> const Byte *cur, CLzRef *son,
>
> - UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> - UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit,
> UInt32 size, UInt32 *posRes)
>
> +MY_NO_INLINE
>
> +static UInt32 *GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte
> *cur, CLzRef *son,
>
> + size_t _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
>
> + UInt32 *distances, UInt32 _maxLen, const UInt32 *hash, const UInt32
> *limit, UInt32 size, UInt32 *posRes)
>
> {
>
> do
>
> {
>
> - UInt32 *distances = _distances + 1;
>
> - UInt32 curMatch = pos - *hash++;
>
> + UInt32 *_distances = ++distances;
>
> + UInt32 delta = *hash++;
>
>
>
> - CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
>
> - CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
>
> - UInt32 len0 = 0, len1 = 0;
>
> + CLzRef *ptr0 = son + ((size_t)_cyclicBufferPos << 1) + 1;
>
> + CLzRef *ptr1 = son + ((size_t)_cyclicBufferPos << 1);
>
> + unsigned len0 = 0, len1 = 0;
>
> UInt32 cutValue = _cutValue;
>
> - UInt32 maxLen = _maxLen;
>
> - for (;;)
>
> + unsigned maxLen = (unsigned)_maxLen;
>
> +
>
> + /*
>
> + if (size > 1)
>
> {
>
> - UInt32 delta = pos - curMatch;
>
> - if (cutValue-- == 0 || delta >= _cyclicBufferSize)
>
> + UInt32 delta = *hash;
>
> + if (delta < _cyclicBufferSize)
>
> {
>
> - *ptr0 = *ptr1 = kEmptyHashValue;
>
> - break;
>
> + UInt32 cyc1 = _cyclicBufferPos + 1;
>
> + CLzRef *pair = son + ((size_t)(cyc1 - delta + ((delta > cyc1) ?
> _cyclicBufferSize : 0)) << 1);
>
> + Byte b = *(cur + 1 - delta);
>
> + _distances[0] = pair[0];
>
> + _distances[1] = b;
>
> }
>
> + }
>
> + */
>
> + if (cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + }
>
> + else
>
> + for(;;)
>
> + {
>
> {
>
> - CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta >
> _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
>
> + CLzRef *pair = son + ((size_t)(_cyclicBufferPos - delta +
> ((_cyclicBufferPos < delta) ? _cyclicBufferSize : 0)) << 1);
>
> const Byte *pb = cur - delta;
>
> - UInt32 len = (len0 < len1 ? len0 : len1);
>
> + unsigned len = (len0 < len1 ? len0 : len1);
>
> + UInt32 pair0 = *pair;
>
> if (pb[len] == cur[len])
>
> {
>
> if (++len != lenLimit && pb[len] == cur[len])
>
> @@ -272,54 +291,66 @@ static Int32 NO_INLINE GetMatchesSpecN(UInt32
> lenLimit, UInt32 pos, const Byte *
> break;
>
> if (maxLen < len)
>
> {
>
> - *distances++ = maxLen = len;
>
> + maxLen = len;
>
> + *distances++ = (UInt32)len;
>
> *distances++ = delta - 1;
>
> if (len == lenLimit)
>
> {
>
> - *ptr1 = pair[0];
>
> - *ptr0 = pair[1];
>
> + UInt32 pair1 = pair[1];
>
> + *ptr1 = pair0;
>
> + *ptr0 = pair1;
>
> break;
>
> }
>
> }
>
> }
>
> - if (pb[len] < cur[len])
>
> - {
>
> - *ptr1 = curMatch;
>
> - ptr1 = pair + 1;
>
> - curMatch = *ptr1;
>
> - len1 = len;
>
> - }
>
> - else
>
> {
>
> - *ptr0 = curMatch;
>
> - ptr0 = pair;
>
> - curMatch = *ptr0;
>
> - len0 = len;
>
> + UInt32 curMatch = pos - delta;
>
> + // delta = pos - *pair;
>
> + // delta = pos - pair[((UInt32)pb[len] - (UInt32)cur[len]) >>
31];
>
> + if (pb[len] < cur[len])
>
> + {
>
> + delta = pos - pair[1];
>
> + *ptr1 = curMatch;
>
> + ptr1 = pair + 1;
>
> + len1 = len;
>
> + }
>
> + else
>
> + {
>
> + delta = pos - *pair;
>
> + *ptr0 = curMatch;
>
> + ptr0 = pair;
>
> + len0 = len;
>
> + }
>
> }
>
> }
>
> + if (--cutValue == 0 || delta >= _cyclicBufferSize)
>
> + {
>
> + *ptr0 = *ptr1 = kEmptyHashValue;
>
> + break;
>
> + }
>
> }
>
> pos++;
>
> _cyclicBufferPos++;
>
> cur++;
>
> {
>
> UInt32 num = (UInt32)(distances - _distances);
>
> - *_distances = num - 1;
>
> - _distances += num;
>
> - limit -= num;
>
> + _distances[-1] = num;
>
> }
>
> }
>
> - while (limit > 0 && --size != 0);
>
> + while (distances < limit && --size != 0);
>
> *posRes = pos;
>
> - return limit;
>
> + return distances;
>
> }
>
>
>
> #endif
>
>
>
> +
>
> +
>
> static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances)
>
> {
>
> UInt32 numProcessed = 0;
>
> UInt32 curPos = 2;
>
> - UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2);
>
> + UInt32 limit = kMtBtBlockSize - (p->matchMaxLen * 2); // * 2
>
>
>
> distances[1] = p->hashNumAvail;
>
>
>
> @@ -369,8 +400,10 @@ static void BtGetMatches(CMatchFinderMt *p,
> UInt32 *distances)
> #else
>
> {
>
> UInt32 posRes;
>
> - curPos = limit - GetMatchesSpecN(lenLimit, pos, p->buffer,
p->son,
> cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> - distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos, (Int32)(limit - curPos), size, &posRes);
>
> + curPos = (UInt32)(GetMatchesSpecN(lenLimit, pos, p->buffer,
> p->son, cyclicBufferPos, p->cyclicBufferSize, p->cutValue,
>
> + distances + curPos, p->numHashBytes - 1, p->hashBuf +
> p->hashBufPos,
>
> + distances + limit,
>
> + size, &posRes) - distances);
>
> p->hashBufPos += posRes - pos;
>
> cyclicBufferPos += posRes - pos;
>
> p->buffer += posRes - pos;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> index 3d86c788f3..fdd17008c2 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h
> @@ -1,5 +1,5 @@
> /* LzFindMt.h -- multithreaded Match finder for LZ algorithms
>
> -2017-04-03 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #ifndef __LZ_FIND_MT_H
>
> #define __LZ_FIND_MT_H
>
> @@ -19,10 +19,10 @@ EXTERN_C_BEGIN
>
>
> typedef struct _CMtSync
>
> {
>
> - Bool wasCreated;
>
> - Bool needStart;
>
> - Bool exit;
>
> - Bool stopWriting;
>
> + BoolInt wasCreated;
>
> + BoolInt needStart;
>
> + BoolInt exit;
>
> + BoolInt stopWriting;
>
>
>
> CThread thread;
>
> CAutoResetEvent canStart;
>
> @@ -30,8 +30,8 @@ typedef struct _CMtSync
> CAutoResetEvent wasStopped;
>
> CSemaphore freeSemaphore;
>
> CSemaphore filledSemaphore;
>
> - Bool csWasInitialized;
>
> - Bool csWasEntered;
>
> + BoolInt csWasInitialized;
>
> + BoolInt csWasEntered;
>
> CCriticalSection cs;
>
> UInt32 numProcessedBlocks;
>
> } CMtSync;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> index 962b94bb63..4d1576419f 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c
> @@ -1,13 +1,13 @@
> /* LzmaDec.c -- LZMA Decoder
>
> -2018-02-28 : Igor Pavlov : Public domain */
>
> +2018-07-04 : Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> +#include <string.h>
>
> +
>
> /* #include "CpuArch.h" */
>
> #include "LzmaDec.h"
>
>
>
> -#include <string.h>
>
> -
>
> #define kNumTopBits 24
>
> #define kTopValue ((UInt32)1 << kNumTopBits)
>
>
>
> @@ -19,7 +19,7 @@
>
>
> #define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code <<
8)
> | (*buf++); }
>
>
>
> -#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt +
> ((kBitModelTotal - ttt) >> kNumMoveBits));
>
> #define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt
> - (ttt >> kNumMoveBits));
>
> #define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
>
> @@ -66,7 +66,7 @@
>
>
> #define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit)
> return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
>
>
>
> -#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * ttt; if (code < bound)
>
> +#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >>
> kNumBitModelTotalBits) * (UInt32)ttt; if (code < bound)
>
> #define UPDATE_0_CHECK range = bound;
>
> #define UPDATE_1_CHECK range -= bound; code -= bound;
>
> #define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
>
> @@ -539,7 +539,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> curLen = ((rem < len) ? (unsigned)rem : len);
>
> pos = dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0);
>
>
>
> - processedPos += curLen;
>
> + processedPos += (UInt32)curLen;
>
>
>
> len -= curLen;
>
> if (curLen <= dicBufSize - pos)
>
> @@ -547,7 +547,7 @@ int MY_FAST_CALL LZMA_DECODE_REAL(CLzmaDec
> *p, SizeT limit, const Byte *bufLimit
> Byte *dest = dic + dicPos;
>
> ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
>
> const Byte *lim = dest + curLen;
>
> - dicPos += curLen;
>
> + dicPos += (SizeT)curLen;
>
> do
>
> *(dest) = (Byte)*(dest + src);
>
> while (++dest != lim);
>
> @@ -572,14 +572,14 @@ int MY_FAST_CALL
> LZMA_DECODE_REAL(CLzmaDec *p, SizeT limit, const Byte *bufLimit
> p->buf = buf;
>
> p->range = range;
>
> p->code = code;
>
> - p->remainLen = len;
>
> + p->remainLen = (UInt32)len;
>
> p->dicPos = dicPos;
>
> p->processedPos = processedPos;
>
> p->reps[0] = rep0;
>
> p->reps[1] = rep1;
>
> p->reps[2] = rep2;
>
> p->reps[3] = rep3;
>
> - p->state = state;
>
> + p->state = (UInt32)state;
>
>
>
> return SZ_OK;
>
> }
>
> @@ -601,8 +601,8 @@ static void MY_FAST_CALL
> LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
> if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
>
> p->checkDicSize = p->prop.dicSize;
>
>
>
> - p->processedPos += len;
>
> - p->remainLen -= len;
>
> + p->processedPos += (UInt32)len;
>
> + p->remainLen -= (UInt32)len;
>
> while (len != 0)
>
> {
>
> len--;
>
> @@ -850,7 +850,7 @@ static ELzmaDummy LzmaDec_TryDummy(const
> CLzmaDec *p, const Byte *buf, SizeT inS
> }
>
>
>
>
>
> -void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
>
> +void LzmaDec_InitDicAndState(CLzmaDec *p, BoolInt initDic, BoolInt
> initState)
>
> {
>
> p->remainLen = kMatchSpecLenStart + 1;
>
> p->tempBufSize = 0;
>
> @@ -979,10 +979,10 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> p->tempBufSize = rem;
>
> if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
>
> {
>
> - int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
>
> + int dummyRes = LzmaDec_TryDummy(p, p->tempBuf,
> (SizeT)rem);
>
> if (dummyRes == DUMMY_ERROR)
>
> {
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> *status = LZMA_STATUS_NEEDS_MORE_INPUT;
>
> return SZ_OK;
>
> }
>
> @@ -1005,9 +1005,9 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT
> dicLimit, const Byte *src, SizeT *sr
> return SZ_ERROR_FAIL; /* some internal error */
>
> lookAhead -= rem;
>
> }
>
> - (*srcLen) += lookAhead;
>
> + (*srcLen) += (SizeT)lookAhead;
>
> src += lookAhead;
>
> - inSize -= lookAhead;
>
> + inSize -= (SizeT)lookAhead;
>
> p->tempBufSize = 0;
>
> }
>
> }
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> index e281716fee..14086fc4f9 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c
> @@ -1,5 +1,5 @@
> /* LzmaEnc.c -- LZMA Encoder
>
> -2018-04-29 : Igor Pavlov : Public domain */
>
> +2019-01-10: Igor Pavlov : Public domain */
>
>
>
> #include "Precomp.h"
>
>
>
> @@ -38,6 +38,8 @@ static unsigned g_STAT_OFFSET = 0;
> #define kNumBitPriceShiftBits 4
>
> #define kBitPrice (1 << kNumBitPriceShiftBits)
>
>
>
> +#define REP_LEN_COUNT 64
>
> +
>
> void LzmaEncProps_Init(CLzmaEncProps *p)
>
> {
>
> p->level = 5;
>
> @@ -183,8 +185,11 @@ typedef struct
> } COptimal;
>
>
>
>
>
> -#define kNumOpts (1 << 12)
>
> -#define kPackReserve (1 + kNumOpts * 2)
>
> +// 18.06
>
> +#define kNumOpts (1 << 11)
>
> +#define kPackReserve (kNumOpts * 8)
>
> +// #define kNumOpts (1 << 12)
>
> +// #define kPackReserve (1 + kNumOpts * 2)
>
>
>
> #define kNumLenToPosStates 4
>
> #define kNumPosSlotBits 6
>
> @@ -236,10 +241,18 @@ typedef struct
> typedef struct
>
> {
>
> unsigned tableSize;
>
> - unsigned counters[LZMA_NUM_PB_STATES_MAX];
>
> UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
>
> + // UInt32 prices1[LZMA_NUM_PB_STATES_MAX][kLenNumLowSymbols *
> 2];
>
> + // UInt32 prices2[kLenNumSymbolsTotal];
>
> } CLenPriceEnc;
>
>
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices[posState][(size_t)(len) - LZMA_MATCH_LEN_MIN])
>
> +
>
> +/*
>
> +#define GET_PRICE_LEN(p, posState, len) \
>
> + ((p)->prices2[(size_t)(len) - 2] + ((p)->prices1[posState][((len) -
2) &
> (kLenNumLowSymbols * 2 - 1)] & (((len) - 2 - kLenNumLowSymbols * 2) >>
9)))
>
> +*/
>
>
>
> typedef struct
>
> {
>
> @@ -308,16 +321,18 @@ typedef struct
> unsigned lc, lp, pb;
>
> unsigned lclp;
>
>
>
> - Bool fastMode;
>
> - Bool writeEndMark;
>
> - Bool finished;
>
> - Bool multiThread;
>
> - Bool needInit;
>
> + BoolInt fastMode;
>
> + BoolInt writeEndMark;
>
> + BoolInt finished;
>
> + BoolInt multiThread;
>
> + BoolInt needInit;
>
> + // BoolInt _maxMode;
>
>
>
> UInt64 nowPos64;
>
>
>
> unsigned matchPriceCount;
>
> - unsigned alignPriceCount;
>
> + // unsigned alignPriceCount;
>
> + int repLenEncCounter;
>
>
>
> unsigned distTableSize;
>
>
>
> @@ -325,7 +340,7 @@ typedef struct
> SRes result;
>
>
>
> #ifndef _7ZIP_ST
>
> - Bool mtMode;
>
> + BoolInt mtMode;
>
> // begin of CMatchFinderMt is used in LZ thread
>
> CMatchFinderMt matchFinderMt;
>
> // end of CMatchFinderMt is used in BT and HASH threads
>
> @@ -458,6 +473,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const
> CLzmaEncProps *props2)
> p->lp = props.lp;
>
> p->pb = props.pb;
>
> p->fastMode = (props.algo == 0);
>
> + // p->_maxMode = True;
>
> p->matchFinderBase.btMode = (Byte)(props.btMode ? 1 : 0);
>
> {
>
> unsigned numHashBytes = 4;
>
> @@ -618,9 +634,9 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #ifdef _LZMA_ENC_USE_BRANCH
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> RC_BIT_PRE(p, prob) \
>
> - if (symbol == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> + if (bit == 0) { range = newBound; ttt += (kBitModelTotal - ttt) >>
> kNumMoveBits; } \
>
> else { (p)->low += newBound; range -= newBound; ttt -= ttt >>
> kNumMoveBits; } \
>
> *(prob) = (CLzmaProb)ttt; \
>
> RC_NORM(p) \
>
> @@ -628,15 +644,15 @@ static void RangeEnc_FlushData(CRangeEnc *p)
>
>
> #else
>
>
>
> -#define RC_BIT(p, prob, symbol) { \
>
> +#define RC_BIT(p, prob, bit) { \
>
> UInt32 mask; \
>
> RC_BIT_PRE(p, prob) \
>
> - mask = 0 - (UInt32)symbol; \
>
> + mask = 0 - (UInt32)bit; \
>
> range &= mask; \
>
> mask &= newBound; \
>
> range -= mask; \
>
> (p)->low += mask; \
>
> - mask = (UInt32)symbol - 1; \
>
> + mask = (UInt32)bit - 1; \
>
> range += newBound & mask; \
>
> mask &= (kBitModelTotal - ((1 << kNumMoveBits) - 1)); \
>
> mask += ((1 << kNumMoveBits) - 1); \
>
> @@ -673,42 +689,42 @@ static void RangeEnc_EncodeBit_0(CRangeEnc *p,
> CLzmaProb *prob)
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
>
> +static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 sym)
>
> {
>
> UInt32 range = p->range;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - // RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
>
> - CLzmaProb *prob = probs + (symbol >> 8);
>
> - UInt32 bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> + // RangeEnc_EncodeBit(p, probs + (sym >> 8), (sym >> 7) & 1);
>
> + CLzmaProb *prob = probs + (sym >> 8);
>
> + UInt32 bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> -static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte)
>
> +static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs,
> UInt32 sym, UInt32 matchByte)
>
> {
>
> UInt32 range = p->range;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> CLzmaProb *prob;
>
> UInt32 bit;
>
> matchByte <<= 1;
>
> - // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol
>>
> 8)), (symbol >> 7) & 1);
>
> - prob = probs + (offs + (matchByte & offs) + (symbol >> 8));
>
> - bit = (symbol >> 7) & 1;
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + // RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (sym >>
> 8)), (sym >> 7) & 1);
>
> + prob = probs + (offs + (matchByte & offs) + (sym >> 8));
>
> + bit = (sym >> 7) & 1;
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> RC_BIT(p, prob, bit);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> p->range = range;
>
> }
>
>
>
> @@ -739,11 +755,11 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> }
>
>
>
>
>
> -#define GET_PRICE(prob, symbol) \
>
> - p->ProbPrices[((prob) ^ (unsigned)(((-(int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICE(prob, bit) \
>
> + p->ProbPrices[((prob) ^ (unsigned)(((-(int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> -#define GET_PRICEa(prob, symbol) \
>
> - ProbPrices[((prob) ^ (unsigned)((-((int)(symbol))) & (kBitModelTotal
-
> 1))) >> kNumMoveReducingBits];
>
> +#define GET_PRICEa(prob, bit) \
>
> + ProbPrices[((prob) ^ (unsigned)((-((int)(bit))) & (kBitModelTotal -
1))) >>
> kNumMoveReducingBits];
>
>
>
> #define GET_PRICE_0(prob) p->ProbPrices[(prob) >>
> kNumMoveReducingBits]
>
> #define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1))
>>
> kNumMoveReducingBits]
>
> @@ -752,48 +768,48 @@ static void LzmaEnc_InitPriceTables(CProbPrice
> *ProbPrices)
> #define GET_PRICEa_1(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >>
> kNumMoveReducingBits]
>
>
>
>
>
> -static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol,
const
> CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 sym, const
> CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> - price += GET_PRICEa(probs[symbol], bit);
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> }
>
> - while (symbol >= 2);
>
> + while (sym >= 2);
>
> return price;
>
> }
>
>
>
>
>
> -static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> symbol, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> +static UInt32 LitEnc_Matched_GetPrice(const CLzmaProb *probs, UInt32
> sym, UInt32 matchByte, const CProbPrice *ProbPrices)
>
> {
>
> UInt32 price = 0;
>
> UInt32 offs = 0x100;
>
> - symbol |= 0x100;
>
> + sym |= 0x100;
>
> do
>
> {
>
> matchByte <<= 1;
>
> - price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)],
> (symbol >> 7) & 1);
>
> - symbol <<= 1;
>
> - offs &= ~(matchByte ^ symbol);
>
> + price += GET_PRICEa(probs[offs + (matchByte & offs) + (sym >> 8)],
> (sym >> 7) & 1);
>
> + sym <<= 1;
>
> + offs &= ~(matchByte ^ sym);
>
> }
>
> - while (symbol < 0x10000);
>
> + while (sym < 0x10000);
>
> return price;
>
> }
>
>
>
>
>
> -static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, UInt32 symbol)
>
> +static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs,
> unsigned numBits, unsigned sym)
>
> {
>
> UInt32 range = rc->range;
>
> unsigned m = 1;
>
> do
>
> {
>
> UInt32 ttt, newBound;
>
> - unsigned bit = symbol & 1;
>
> + unsigned bit = sym & 1;
>
> // RangeEnc_EncodeBit(rc, probs + m, bit);
>
> - symbol >>= 1;
>
> + sym >>= 1;
>
> RC_BIT(rc, probs + m, bit);
>
> m = (m << 1) | bit;
>
> }
>
> @@ -812,37 +828,37 @@ static void LenEnc_Init(CLenEnc *p)
> p->high[i] = kProbInitValue;
>
> }
>
>
>
> -static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned symbol,
> unsigned posState)
>
> +static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, unsigned sym,
> unsigned posState)
>
> {
>
> UInt32 range, ttt, newBound;
>
> CLzmaProb *probs = p->low;
>
> range = rc->range;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols)
>
> + if (sym >= kLenNumLowSymbols)
>
> {
>
> RC_BIT_1(rc, probs);
>
> probs += kLenNumLowSymbols;
>
> RC_BIT_PRE(rc, probs);
>
> - if (symbol >= kLenNumLowSymbols * 2)
>
> + if (sym >= kLenNumLowSymbols * 2)
>
> {
>
> RC_BIT_1(rc, probs);
>
> rc->range = range;
>
> - // RcTree_Encode(rc, p->high, kLenNumHighBits, symbol -
> kLenNumLowSymbols * 2);
>
> - LitEnc_Encode(rc, p->high, symbol - kLenNumLowSymbols * 2);
>
> + // RcTree_Encode(rc, p->high, kLenNumHighBits, sym -
> kLenNumLowSymbols * 2);
>
> + LitEnc_Encode(rc, p->high, sym - kLenNumLowSymbols * 2);
>
> return;
>
> }
>
> - symbol -= kLenNumLowSymbols;
>
> + sym -= kLenNumLowSymbols;
>
> }
>
>
>
> - // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, symbol);
>
> + // RcTree_Encode(rc, probs + (posState << kLenNumLowBits),
> kLenNumLowBits, sym);
>
> {
>
> unsigned m;
>
> unsigned bit;
>
> RC_BIT_0(rc, probs);
>
> probs += (posState << (1 + kLenNumLowBits));
>
> - bit = (symbol >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) +
bit;
>
> - bit = (symbol >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) +
bit;
>
> - bit = symbol & 1; RC_BIT(rc, probs + m, bit);
>
> + bit = (sym >> 2) ; RC_BIT(rc, probs + 1, bit); m = (1 << 1) + bit;
>
> + bit = (sym >> 1) & 1; RC_BIT(rc, probs + m, bit); m = (m << 1) + bit;
>
> + bit = sym & 1; RC_BIT(rc, probs + m, bit);
>
> rc->range = range;
>
> }
>
> }
>
> @@ -863,50 +879,93 @@ static void SetPrices_3(const CLzmaProb *probs,
> UInt32 startPrice, UInt32 *price
> }
>
>
>
>
>
> -MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTable(
>
> - CLenPriceEnc *p, unsigned posState,
>
> +MY_NO_INLINE static void MY_FAST_CALL LenPriceEnc_UpdateTables(
>
> + CLenPriceEnc *p,
>
> + unsigned numPosStates,
>
> const CLenEnc *enc,
>
> const CProbPrice *ProbPrices)
>
> {
>
> - // int y; for (y = 0; y < 100; y++) {
>
> - UInt32 a;
>
> - unsigned i, numSymbols;
>
> + UInt32 b;
>
> +
>
> + {
>
> + unsigned prob = enc->low[0];
>
> + UInt32 a, c;
>
> + unsigned posState;
>
> + b = GET_PRICEa_1(prob);
>
> + a = GET_PRICEa_0(prob);
>
> + c = b + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (posState = 0; posState < numPosStates; posState++)
>
> + {
>
> + UInt32 *prices = p->prices[posState];
>
> + const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> + SetPrices_3(probs, a, prices, ProbPrices);
>
> + SetPrices_3(probs + kLenNumLowSymbols, c, prices +
> kLenNumLowSymbols, ProbPrices);
>
> + }
>
> + }
>
>
>
> - UInt32 *prices = p->prices[posState];
>
> + /*
>
> {
>
> - const CLzmaProb *probs = enc->low + (posState << (1 +
> kLenNumLowBits));
>
> - SetPrices_3(probs, GET_PRICEa_0(enc->low[0]), prices, ProbPrices);
>
> + unsigned i;
>
> + UInt32 b;
>
> + a = GET_PRICEa_0(enc->low[0]);
>
> + for (i = 0; i < kLenNumLowSymbols; i++)
>
> + p->prices2[i] = a;
>
> a = GET_PRICEa_1(enc->low[0]);
>
> - SetPrices_3(probs + kLenNumLowSymbols, a +
> GET_PRICEa_0(enc->low[kLenNumLowSymbols]), prices +
> kLenNumLowSymbols, ProbPrices);
>
> + b = a + GET_PRICEa_0(enc->low[kLenNumLowSymbols]);
>
> + for (i = kLenNumLowSymbols; i < kLenNumLowSymbols * 2; i++)
>
> + p->prices2[i] = b;
>
> a += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> }
>
> - numSymbols = p->tableSize;
>
> - p->counters[posState] = numSymbols;
>
> - for (i = kLenNumLowSymbols * 2; i < numSymbols; i += 1)
>
> + */
>
> +
>
> + // p->counter = numSymbols;
>
> + // p->counter = 64;
>
> +
>
> {
>
> - prices[i] = a +
>
> - // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> - LitEnc_GetPrice(enc->high, i - kLenNumLowSymbols * 2,
> ProbPrices);
>
> - /*
>
> - unsigned sym = (i - kLenNumLowSymbols * 2) >> 1;
>
> - UInt32 price = a + RcTree_GetPrice(enc->high, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> - UInt32 prob = enc->high[(1 << 7) + sym];
>
> - prices[i ] = price + GET_PRICEa_0(prob);
>
> - prices[i + 1] = price + GET_PRICEa_1(prob);
>
> - */
>
> - }
>
> - // }
>
> -}
>
> + unsigned i = p->tableSize;
>
> +
>
> + if (i > kLenNumLowSymbols * 2)
>
> + {
>
> + const CLzmaProb *probs = enc->high;
>
> + UInt32 *prices = p->prices[0] + kLenNumLowSymbols * 2;
>
> + i -= kLenNumLowSymbols * 2 - 1;
>
> + i >>= 1;
>
> + b += GET_PRICEa_1(enc->low[kLenNumLowSymbols]);
>
> + do
>
> + {
>
> + /*
>
> + p->prices2[i] = a +
>
> + // RcTree_GetPrice(enc->high, kLenNumHighBits, i -
> kLenNumLowSymbols * 2, ProbPrices);
>
> + LitEnc_GetPrice(probs, i - kLenNumLowSymbols * 2, ProbPrices);
>
> + */
>
> + // UInt32 price = a + RcTree_GetPrice(probs, kLenNumHighBits - 1,
> sym, ProbPrices);
>
> + unsigned sym = --i + (1 << (kLenNumHighBits - 1));
>
> + UInt32 price = b;
>
> + do
>
> + {
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> + price += GET_PRICEa(probs[sym], bit);
>
> + }
>
> + while (sym >= 2);
>
>
>
> -static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, unsigned
> numPosStates,
>
> - const CLenEnc *enc,
>
> - const CProbPrice *ProbPrices)
>
> -{
>
> - unsigned posState;
>
> - for (posState = 0; posState < numPosStates; posState++)
>
> - LenPriceEnc_UpdateTable(p, posState, enc, ProbPrices);
>
> -}
>
> + {
>
> + unsigned prob = probs[(size_t)i + (1 << (kLenNumHighBits -
1))];
>
> + prices[(size_t)i * 2 ] = price + GET_PRICEa_0(prob);
>
> + prices[(size_t)i * 2 + 1] = price + GET_PRICEa_1(prob);
>
> + }
>
> + }
>
> + while (i);
>
>
>
> + {
>
> + unsigned posState;
>
> + size_t num = (p->tableSize - kLenNumLowSymbols * 2) *
> sizeof(p->prices[0][0]);
>
> + for (posState = 1; posState < numPosStates; posState++)
>
> + memcpy(p->prices[posState] + kLenNumLowSymbols * 2,
> p->prices[0] + kLenNumLowSymbols * 2, num);
>
> + }
>
> + }
>
> + }
>
> +}
>
>
>
> /*
>
> #ifdef SHOW_STAT
>
> @@ -917,7 +976,7 @@ static void LenPriceEnc_UpdateTables(CLenPriceEnc
> *p, unsigned numPosStates,
>
>
> #define MOVE_POS(p, num) { \
>
> p->additionalOffset += (num); \
>
> - p->matchFinder.Skip(p->matchFinderObj, (num)); }
>
> + p->matchFinder.Skip(p->matchFinderObj, (UInt32)(num)); }
>
>
>
>
>
> static unsigned ReadMatchDistances(CLzmaEnc *p, unsigned *numPairsRes)
>
> @@ -954,7 +1013,8 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> const Byte *p2 = p1 + len;
>
> ptrdiff_t dif = (ptrdiff_t)-1 - p->matches[(size_t)numPairs - 1];
>
> const Byte *lim = p1 + numAvail;
>
> - for (; p2 != lim && *p2 == p2[dif]; p2++);
>
> + for (; p2 != lim && *p2 == p2[dif]; p2++)
>
> + {}
>
> return (unsigned)(p2 - p1);
>
> }
>
> }
>
> @@ -977,7 +1037,7 @@ static unsigned ReadMatchDistances(CLzmaEnc *p,
> unsigned *numPairsRes)
> + GET_PRICE_1(p->isRep[state]) \
>
> + GET_PRICE_0(p->isRepG0[state])
>
>
>
> -
>
> +MY_FORCE_INLINE
>
> static UInt32 GetPrice_PureRep(const CLzmaEnc *p, unsigned repIndex,
> size_t state, size_t posState)
>
> {
>
> UInt32 price;
>
> @@ -1011,14 +1071,14 @@ static unsigned Backward(CLzmaEnc *p,
> unsigned cur)
> for (;;)
>
> {
>
> UInt32 dist = p->opt[cur].dist;
>
> - UInt32 len = p->opt[cur].len;
>
> - UInt32 extra = p->opt[cur].extra;
>
> + unsigned len = (unsigned)p->opt[cur].len;
>
> + unsigned extra = (unsigned)p->opt[cur].extra;
>
> cur -= len;
>
>
>
> if (extra)
>
> {
>
> wr--;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> cur -= extra;
>
> len = extra;
>
> if (extra == 1)
>
> @@ -1045,7 +1105,7 @@ static unsigned Backward(CLzmaEnc *p, unsigned
> cur)
>
>
> wr--;
>
> p->opt[wr].dist = dist;
>
> - p->opt[wr].len = len;
>
> + p->opt[wr].len = (UInt32)len;
>
> }
>
> }
>
>
>
> @@ -1102,7 +1162,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> repLens[i] = 0;
>
> continue;
>
> }
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> repLens[i] = len;
>
> if (len > repLens[repMaxIndex])
>
> repMaxIndex = i;
>
> @@ -1111,7 +1172,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (repLens[repMaxIndex] >= p->numFastBytes)
>
> {
>
> unsigned len;
>
> - p->backRes = repMaxIndex;
>
> + p->backRes = (UInt32)repMaxIndex;
>
> len = repLens[repMaxIndex];
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> @@ -1128,8 +1189,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
> +
>
> + last = repLens[repMaxIndex];
>
> + if (last <= mainLen)
>
> + last = mainLen;
>
>
>
> - if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] <
> 2)
>
> + if (last < 2 && curByte != matchByte)
>
> {
>
> p->backRes = MARK_LIT;
>
> return 1;
>
> @@ -1146,13 +1211,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> LitEnc_Matched_GetPrice(probs, curByte, matchByte,
> p->ProbPrices) :
>
> LitEnc_GetPrice(probs, curByte, p->ProbPrices));
>
> }
>
> -
>
> +
>
> MakeAs_Lit(&p->opt[1]);
>
>
>
> matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
>
>
>
> - if (matchByte == curByte)
>
> + // 18.06
>
> + if (matchByte == curByte && repLens[0] == 0)
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p,
> p->state, posState);
>
> if (shortRepPrice < p->opt[1].price)
>
> @@ -1160,16 +1226,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[1].price = shortRepPrice;
>
> MakeAs_ShortRep(&p->opt[1]);
>
> }
>
> + if (last < 2)
>
> + {
>
> + p->backRes = p->opt[1].dist;
>
> + return 1;
>
> + }
>
> }
>
> -
>
> - last = (mainLen >= repLens[repMaxIndex] ? mainLen :
> repLens[repMaxIndex]);
>
> -
>
> - if (last < 2)
>
> - {
>
> - p->backRes = p->opt[1].dist;
>
> - return 1;
>
> - }
>
> -
>
> +
>
> p->opt[1].len = 1;
>
>
>
> p->opt[0].reps[0] = reps[0];
>
> @@ -1177,13 +1240,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> p->opt[0].reps[2] = reps[2];
>
> p->opt[0].reps[3] = reps[3];
>
>
>
> - {
>
> - unsigned len = last;
>
> - do
>
> - p->opt[len--].price = kInfinityPrice;
>
> - while (len >= 2);
>
> - }
>
> -
>
> // ---------- REP ----------
>
>
>
> for (i = 0; i < LZMA_NUM_REPS; i++)
>
> @@ -1195,13 +1251,13 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price = repMatchPrice + GetPrice_PureRep(p, i, p->state, posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)repLen - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc, posState,
> repLen);
>
> COptimal *opt = &p->opt[repLen];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = repLen;
>
> - opt->dist = i;
>
> + opt->len = (UInt32)repLen;
>
> + opt->dist = (UInt32)i;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1211,38 +1267,41 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> // ---------- MATCH ----------
>
> {
>
> - unsigned len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
>
> + unsigned len = repLens[0] + 1;
>
> if (len <= mainLen)
>
> {
>
> unsigned offs = 0;
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[p->state]);
>
>
>
> - while (len > matches[offs])
>
> - offs += 2;
>
> + if (len < 2)
>
> + len = 2;
>
> + else
>
> + while (len > matches[offs])
>
> + offs += 2;
>
>
>
> for (; ; len++)
>
> {
>
> COptimal *opt;
>
> UInt32 dist = matches[(size_t)offs + 1];
>
> - UInt32 price2 = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice +
> GET_PRICE_LEN(&p->lenEnc, posState, len);
>
> unsigned lenToPosState = GetLenToPosState(len);
>
>
>
> if (dist < kNumFullDistances)
>
> - price2 += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> else
>
> {
>
> unsigned slot;
>
> GetPosSlot2(dist, slot);
>
> - price2 += p->alignPrices[dist & kAlignMask];
>
> - price2 += p->posSlotPrices[lenToPosState][slot];
>
> + price += p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenToPosState][slot];
>
> }
>
>
>
> opt = &p->opt[len];
>
>
>
> - if (price2 < opt->price)
>
> + if (price < opt->price)
>
> {
>
> - opt->price = price2;
>
> - opt->len = len;
>
> + opt->price = price;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> @@ -1277,16 +1336,43 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (;;)
>
> {
>
> - UInt32 numAvail, numAvailFull;
>
> + unsigned numAvail;
>
> + UInt32 numAvailFull;
>
> unsigned newLen, numPairs, prev, state, posState, startLen;
>
> - UInt32 curPrice, litPrice, matchPrice, repMatchPrice;
>
> - Bool nextIsLit;
>
> + UInt32 litPrice, matchPrice, repMatchPrice;
>
> + BoolInt nextIsLit;
>
> Byte curByte, matchByte;
>
> const Byte *data;
>
> COptimal *curOpt, *nextOpt;
>
>
>
> if (++cur == last)
>
> - return Backward(p, cur);
>
> + break;
>
> +
>
> + // 18.06
>
> + if (cur >= kNumOpts - 64)
>
> + {
>
> + unsigned j, best;
>
> + UInt32 price = p->opt[cur].price;
>
> + best = cur;
>
> + for (j = cur + 1; j <= last; j++)
>
> + {
>
> + UInt32 price2 = p->opt[j].price;
>
> + if (price >= price2)
>
> + {
>
> + price = price2;
>
> + best = j;
>
> + }
>
> + }
>
> + {
>
> + unsigned delta = best - cur;
>
> + if (delta != 0)
>
> + {
>
> + MOVE_POS(p, delta);
>
> + }
>
> + }
>
> + cur = best;
>
> + break;
>
> + }
>
>
>
> newLen = ReadMatchDistances(p, &numPairs);
>
>
>
> @@ -1294,15 +1380,24 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> p->numPairs = numPairs;
>
> p->longestMatchLen = newLen;
>
> - return Backward(p, cur);
>
> + break;
>
> }
>
>
>
> curOpt = &p->opt[cur];
>
> +
>
> + position++;
>
> +
>
> + // we need that check here, if skip_items in p->opt are possible
>
> + /*
>
> + if (curOpt->price >= kInfinityPrice)
>
> + continue;
>
> + */
>
> +
>
> prev = cur - curOpt->len;
>
> -
>
> +
>
> if (curOpt->len == 1)
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (IsShortRep(curOpt))
>
> state = kShortRepNextStates[state];
>
> else
>
> @@ -1316,14 +1411,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> if (curOpt->extra)
>
> {
>
> - prev -= curOpt->extra;
>
> + prev -= (unsigned)curOpt->extra;
>
> state = kState_RepAfterLit;
>
> if (curOpt->extra == 1)
>
> - state = (dist < LZMA_NUM_REPS) ? kState_RepAfterLit :
> kState_MatchAfterLit;
>
> + state = (dist < LZMA_NUM_REPS ? kState_RepAfterLit :
> kState_MatchAfterLit);
>
> }
>
> else
>
> {
>
> - state = p->opt[prev].state;
>
> + state = (unsigned)p->opt[prev].state;
>
> if (dist < LZMA_NUM_REPS)
>
> state = kRepNextStates[state];
>
> else
>
> @@ -1379,7 +1474,6 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> curByte = *data;
>
> matchByte = *(data - reps[0]);
>
>
>
> - position++;
>
> posState = (position & p->pbMask);
>
>
>
> /*
>
> @@ -1391,13 +1485,25 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> < MATCH [ : LIT : REP_0 ]
>
> */
>
>
>
> - curPrice = curOpt->price;
>
> - litPrice = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
>
> + {
>
> + UInt32 curPrice = curOpt->price;
>
> + unsigned prob = p->isMatch[state][posState];
>
> + matchPrice = curPrice + GET_PRICE_1(prob);
>
> + litPrice = curPrice + GET_PRICE_0(prob);
>
> + }
>
>
>
> nextOpt = &p->opt[(size_t)cur + 1];
>
> nextIsLit = False;
>
>
>
> - // if (litPrice >= nextOpt->price) litPrice = 0; else // 18.new
>
> + // here we can allow skip_items in p->opt, if we don't check
> (nextOpt->price < kInfinityPrice)
>
> + // 18.new.06
>
> + if ((nextOpt->price < kInfinityPrice
>
> + // && !IsLitState(state)
>
> + && matchByte == curByte)
>
> + || litPrice > nextOpt->price
>
> + )
>
> + litPrice = 0;
>
> + else
>
> {
>
> const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
>
> litPrice += (!IsLitState(state) ?
>
> @@ -1413,21 +1519,32 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
>
> repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
>
>
>
> + numAvailFull = p->numAvail;
>
> + {
>
> + unsigned temp = kNumOpts - 1 - cur;
>
> + if (numAvailFull > temp)
>
> + numAvailFull = (UInt32)temp;
>
> + }
>
> +
>
> + // 18.06
>
> // ---------- SHORT_REP ----------
>
> - // if (IsLitState(state)) // 18.new
>
> + if (IsLitState(state)) // 18.new
>
> if (matchByte == curByte)
>
> - // if (repMatchPrice < nextOpt->price) // 18.new
>
> - if (nextOpt->len < 2
>
> + if (repMatchPrice < nextOpt->price) // 18.new
>
> + // if (numAvailFull < 2 || data[1] != *(data - reps[0] + 1))
>
> + if (
>
> + // nextOpt->price >= kInfinityPrice ||
>
> + nextOpt->len < 2 // we can check nextOpt->len, if skip items
> are not allowed in p->opt
>
> || (nextOpt->dist != 0
>
> - && nextOpt->extra <= 1 // 17.old
>
> - ))
>
> + // && nextOpt->extra <= 1 // 17.old
>
> + )
>
> + )
>
> {
>
> UInt32 shortRepPrice = repMatchPrice + GetPrice_ShortRep(p, state,
> posState);
>
> - if (shortRepPrice <= nextOpt->price) // 17.old
>
> - // if (shortRepPrice < nextOpt->price) // 18.new
>
> + // if (shortRepPrice <= nextOpt->price) // 17.old
>
> + if (shortRepPrice < nextOpt->price) // 18.new
>
> {
>
> nextOpt->price = shortRepPrice;
>
> nextOpt->len = 1;
>
> @@ -1436,13 +1553,6 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
>
>
> - numAvailFull = p->numAvail;
>
> - {
>
> - UInt32 temp = kNumOpts - 1 - cur;
>
> - if (numAvailFull > temp)
>
> - numAvailFull = temp;
>
> - }
>
> -
>
> if (numAvailFull < 2)
>
> continue;
>
> numAvail = (numAvailFull <= p->numFastBytes ? numAvailFull :
> p->numFastBytes);
>
> @@ -1451,9 +1561,8 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
>
>
> // ---------- LIT : REP_0 ----------
>
>
>
> - if (
>
> - // litPrice != 0 && // 18.new
>
> - !nextIsLit
>
> + if (!nextIsLit
>
> + && litPrice != 0 // 18.new
>
> && matchByte != curByte
>
> && numAvailFull > 2)
>
> {
>
> @@ -1465,8 +1574,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
> for (len = 3; len < limit && data[len] == data2[len]; len++)
>
> - {
>
> - }
>
> + {}
>
>
>
> {
>
> unsigned state2 = kLiteralNextStates[state];
>
> @@ -1474,8 +1582,9 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> UInt32 price = litPrice + GetPrice_Rep_0(p, state2, posState2);
>
> {
>
> unsigned offset = cur + len;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
>
>
> // do
>
> {
>
> @@ -1483,14 +1592,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> COptimal *opt;
>
> len--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = 0;
>
> opt->extra = 1;
>
> }
>
> @@ -1502,6 +1611,7 @@ static unsigned GetOptimum(CLzmaEnc *p, UInt32
> position)
> }
>
>
>
> startLen = 2; /* speed optimization */
>
> +
>
> {
>
> // ---------- REP ----------
>
> unsigned repIndex = 0; // 17.old
>
> @@ -1514,24 +1624,28 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
>
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
>
>
> // if (len < startLen) continue; // 18.new: speed optimization
>
>
>
> - while (last < cur + len)
>
> - p->opt[++last].price = kInfinityPrice;
>
> + {
>
> + unsigned offset = cur + len;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
> {
>
> unsigned len2 = len;
>
> price = repMatchPrice + GetPrice_PureRep(p, repIndex, state,
> posState);
>
> do
>
> {
>
> - UInt32 price2 = price +
> p->repLenEnc.prices[posState][(size_t)len2 - 2];
>
> + UInt32 price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState, len2);
>
> COptimal *opt = &p->opt[cur + len2];
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> - opt->dist = repIndex;
>
> + opt->len = (UInt32)len2;
>
> + opt->dist = (UInt32)repIndex;
>
> opt->extra = 0;
>
> }
>
> }
>
> @@ -1551,15 +1665,14 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> -
>
> - len2 -= len;
>
> - if (len2 >= 3)
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> {
>
> unsigned state2 = kRepNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> - price +=
>
> - p->repLenEnc.prices[posState][(size_t)len - 2]
>
> + price += GET_PRICE_LEN(&p->repLenEnc, posState, len)
>
> + GET_PRICE_0(p->isMatch[state2][posState2])
>
> + LitEnc_Matched_GetPrice(LIT_PROBS(position + len,
> data[(size_t)len - 1]),
>
> data[len], data2[len], p->ProbPrices);
>
> @@ -1570,31 +1683,40 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
>
>
> price += GetPrice_Rep_0(p, state2, posState2);
>
> +
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
> +
>
> + len2 -= len;
>
> + // if (len2 >= 3)
>
> + {
>
> {
>
> unsigned offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> - unsigned price2;
>
> + UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
>
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> - opt->dist = repIndex;
>
> + opt->dist = (UInt32)repIndex;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> }
>
> + }
>
> }
>
> }
>
> }
>
> @@ -1606,17 +1728,23 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> {
>
> newLen = numAvail;
>
> for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
>
> - matches[numPairs] = newLen;
>
> + matches[numPairs] = (UInt32)newLen;
>
> numPairs += 2;
>
> }
>
>
>
> + // startLen = 2; /* speed optimization */
>
> +
>
> if (newLen >= startLen)
>
> {
>
> UInt32 normalMatchPrice = matchPrice +
> GET_PRICE_0(p->isRep[state]);
>
> UInt32 dist;
>
> unsigned offs, posSlot, len;
>
> - while (last < cur + newLen)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + {
>
> + unsigned offset = cur + newLen;
>
> + if (last < offset)
>
> + last = offset;
>
> + }
>
>
>
> offs = 0;
>
> while (startLen > matches[offs])
>
> @@ -1628,27 +1756,29 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
>
>
> for (len = /*2*/ startLen; ; len++)
>
> {
>
> - UInt32 price = normalMatchPrice +
> p->lenEnc.prices[posState][(size_t)len - LZMA_MATCH_LEN_MIN];
>
> + UInt32 price = normalMatchPrice + GET_PRICE_LEN(&p->lenEnc,
> posState, len);
>
> {
>
> COptimal *opt;
>
> - unsigned lenToPosState = len - 2; lenToPosState =
> GetLenToPosState2(lenToPosState);
>
> + unsigned lenNorm = len - 2;
>
> + lenNorm = GetLenToPosState2(lenNorm);
>
> if (dist < kNumFullDistances)
>
> - price += p->distancesPrices[lenToPosState][dist &
> (kNumFullDistances - 1)];
>
> + price += p->distancesPrices[lenNorm][dist &
> (kNumFullDistances - 1)];
>
> else
>
> - price += p->posSlotPrices[lenToPosState][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
> + price += p->posSlotPrices[lenNorm][posSlot] +
> p->alignPrices[dist & kAlignMask];
>
>
>
> opt = &p->opt[cur + len];
>
> if (price < opt->price)
>
> {
>
> opt->price = price;
>
> - opt->len = len;
>
> + opt->len = (UInt32)len;
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> opt->extra = 0;
>
> }
>
> }
>
>
>
> - if (/*_maxMode && */ len == matches[offs])
>
> + if (len == matches[offs])
>
> {
>
> + // if (p->_maxMode) {
>
> // MATCH : LIT : REP_0
>
>
>
> const Byte *data2 = data - dist - 1;
>
> @@ -1657,11 +1787,17 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> if (limit > numAvailFull)
>
> limit = numAvailFull;
>
>
>
> - for (; len2 < limit && data[len2] == data2[len2]; len2++);
>
> + len2 += 2;
>
> + if (len2 <= limit)
>
> + if (data[len2 - 2] == data2[len2 - 2])
>
> + if (data[len2 - 1] == data2[len2 - 1])
>
> + {
>
> + for (; len2 < limit && data[len2] == data2[len2]; len2++)
>
> + {}
>
>
>
> len2 -= len;
>
>
>
> - if (len2 >= 3)
>
> + // if (len2 >= 3)
>
> {
>
> unsigned state2 = kMatchNextStates[state];
>
> unsigned posState2 = (position + len) & p->pbMask;
>
> @@ -1677,27 +1813,30 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> price += GetPrice_Rep_0(p, state2, posState2);
>
>
>
> offset = cur + len + len2;
>
> - while (last < offset)
>
> - p->opt[++last].price = kInfinityPrice;
>
> +
>
> + if (last < offset)
>
> + last = offset;
>
> // do
>
> {
>
> UInt32 price2;
>
> COptimal *opt;
>
> len2--;
>
> // price2 = price + GetPrice_Len_Rep_0(p, len2, state2,
> posState2);
>
> - price2 = price + p->repLenEnc.prices[posState2][len2 -
> LZMA_MATCH_LEN_MIN];
>
> + price2 = price + GET_PRICE_LEN(&p->repLenEnc,
> posState2, len2);
>
> opt = &p->opt[offset];
>
> // offset--;
>
> if (price2 < opt->price)
>
> {
>
> opt->price = price2;
>
> - opt->len = len2;
>
> + opt->len = (UInt32)len2;
>
> opt->extra = (CExtra)(len + 1);
>
> opt->dist = dist + LZMA_NUM_REPS;
>
> }
>
> }
>
> // while (len2 >= 3);
>
> }
>
> +
>
> + }
>
>
>
> offs += 2;
>
> if (offs == numPairs)
>
> @@ -1709,6 +1848,12 @@ static unsigned GetOptimum(CLzmaEnc *p,
> UInt32 position)
> }
>
> }
>
> }
>
> +
>
> + do
>
> + p->opt[last].price = kInfinityPrice;
>
> + while (--last);
>
> +
>
> + return Backward(p, cur);
>
> }
>
>
>
>
>
> @@ -1735,6 +1880,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> p->backRes = MARK_LIT;
>
> if (numAvail < 2)
>
> return 1;
>
> + // if (mainLen < 2 && p->state == 0) return 1; // 18.06.notused
>
> if (numAvail > LZMA_MATCH_LEN_MAX)
>
> numAvail = LZMA_MATCH_LEN_MAX;
>
> data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
>
> @@ -1746,10 +1892,11 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> const Byte *data2 = data - p->reps[i];
>
> if (data[0] != data2[0] || data[1] != data2[1])
>
> continue;
>
> - for (len = 2; len < numAvail && data[len] == data2[len]; len++);
>
> + for (len = 2; len < numAvail && data[len] == data2[len]; len++)
>
> + {}
>
> if (len >= p->numFastBytes)
>
> {
>
> - p->backRes = i;
>
> + p->backRes = (UInt32)i;
>
> MOVE_POS(p, len - 1)
>
> return len;
>
> }
>
> @@ -1793,7 +1940,7 @@ static unsigned GetOptimumFast(CLzmaEnc *p)
> || (repLen + 2 >= mainLen && mainDist >= (1 << 9))
>
> || (repLen + 3 >= mainLen && mainDist >= (1 << 15)))
>
> {
>
> - p->backRes = repIndex;
>
> + p->backRes = (UInt32)repIndex;
>
> MOVE_POS(p, repLen - 1)
>
> return repLen;
>
> }
>
> @@ -1932,23 +2079,22 @@ MY_NO_INLINE static SRes Flush(CLzmaEnc *p,
> UInt32 nowPos)
> }
>
>
>
>
>
> -
>
> -static void FillAlignPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillAlignPrices(CLzmaEnc *p)
>
> {
>
> unsigned i;
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> const CLzmaProb *probs = p->posAlignEncoder;
>
> - p->alignPriceCount = 0;
>
> + // p->alignPriceCount = 0;
>
> for (i = 0; i < kAlignTableSize / 2; i++)
>
> {
>
> UInt32 price = 0;
>
> - unsigned symbol = i;
>
> + unsigned sym = i;
>
> unsigned m = 1;
>
> unsigned bit;
>
> UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[m], bit); m
=
> (m << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[m], bit); m = (m
> << 1) + bit;
>
> prob = probs[m];
>
> p->alignPrices[i ] = price + GET_PRICEa_0(prob);
>
> p->alignPrices[i + 8] = price + GET_PRICEa_1(prob);
>
> @@ -1957,78 +2103,97 @@ static void FillAlignPrices(CLzmaEnc *p)
> }
>
>
>
>
>
> -static void FillDistancesPrices(CLzmaEnc *p)
>
> +MY_NO_INLINE static void FillDistancesPrices(CLzmaEnc *p)
>
> {
>
> + // int y; for (y = 0; y < 100; y++) {
>
> +
>
> UInt32 tempPrices[kNumFullDistances];
>
> - unsigned i, lenToPosState;
>
> + unsigned i, lps;
>
>
>
> const CProbPrice *ProbPrices = p->ProbPrices;
>
> p->matchPriceCount = 0;
>
>
>
> - for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
>
> + for (i = kStartPosModelIndex / 2; i < kNumFullDistances / 2; i++)
>
> {
>
> unsigned posSlot = GetPosSlot1(i);
>
> - unsigned footerBits = ((posSlot >> 1) - 1);
>
> + unsigned footerBits = (posSlot >> 1) - 1;
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> + const CLzmaProb *probs = p->posEncoders + (size_t)base * 2;
>
> // tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base,
> footerBits, i - base, p->ProbPrices);
>
> -
>
> - const CLzmaProb *probs = p->posEncoders + base;
>
> UInt32 price = 0;
>
> unsigned m = 1;
>
> - unsigned symbol = i - base;
>
> + unsigned sym = i;
>
> + unsigned offset = (unsigned)1 << footerBits;
>
> + base += i;
>
> +
>
> + if (footerBits)
>
> do
>
> {
>
> - unsigned bit = symbol & 1;
>
> - symbol >>= 1;
>
> + unsigned bit = sym & 1;
>
> + sym >>= 1;
>
> price += GET_PRICEa(probs[m], bit);
>
> m = (m << 1) + bit;
>
> }
>
> while (--footerBits);
>
> - tempPrices[i] = price;
>
> +
>
> + {
>
> + unsigned prob = probs[m];
>
> + tempPrices[base ] = price + GET_PRICEa_0(prob);
>
> + tempPrices[base + offset] = price + GET_PRICEa_1(prob);
>
> + }
>
> }
>
>
>
> - for (lenToPosState = 0; lenToPosState < kNumLenToPosStates;
> lenToPosState++)
>
> + for (lps = 0; lps < kNumLenToPosStates; lps++)
>
> {
>
> - unsigned posSlot;
>
> - const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
>
> - UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
>
> - unsigned distTableSize = p->distTableSize;
>
> - const CLzmaProb *probs = encoder;
>
> - for (posSlot = 0; posSlot < distTableSize; posSlot += 2)
>
> + unsigned slot;
>
> + unsigned distTableSize2 = (p->distTableSize + 1) >> 1;
>
> + UInt32 *posSlotPrices = p->posSlotPrices[lps];
>
> + const CLzmaProb *probs = p->posSlotEncoder[lps];
>
> +
>
> + for (slot = 0; slot < distTableSize2; slot++)
>
> {
>
> - // posSlotPrices[posSlot] = RcTree_GetPrice(encoder,
> kNumPosSlotBits, posSlot, p->ProbPrices);
>
> - UInt32 price = 0;
>
> + // posSlotPrices[slot] = RcTree_GetPrice(encoder, kNumPosSlotBits,
> slot, p->ProbPrices);
>
> + UInt32 price;
>
> unsigned bit;
>
> - unsigned symbol = (posSlot >> 1) + (1 << (kNumPosSlotBits - 1));
>
> - UInt32 prob;
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - bit = symbol & 1; symbol >>= 1; price += GET_PRICEa(probs[symbol],
> bit);
>
> - prob = probs[(posSlot >> 1) + (1 << (kNumPosSlotBits - 1))];
>
> - posSlotPrices[posSlot ] = price + GET_PRICEa_0(prob);
>
> - posSlotPrices[posSlot + 1] = price + GET_PRICEa_1(prob);
>
> + unsigned sym = slot + (1 << (kNumPosSlotBits - 1));
>
> + unsigned prob;
>
> + bit = sym & 1; sym >>= 1; price = GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + bit = sym & 1; sym >>= 1; price += GET_PRICEa(probs[sym], bit);
>
> + prob = probs[(size_t)slot + (1 << (kNumPosSlotBits - 1))];
>
> + posSlotPrices[(size_t)slot * 2 ] = price + GET_PRICEa_0(prob);
>
> + posSlotPrices[(size_t)slot * 2 + 1] = price + GET_PRICEa_1(prob);
>
> }
>
> - for (posSlot = kEndPosModelIndex; posSlot < distTableSize; posSlot++)
>
> - posSlotPrices[posSlot] += ((UInt32)(((posSlot >> 1) - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> -
>
> +
>
> {
>
> - UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
>
> + UInt32 delta = ((UInt32)((kEndPosModelIndex / 2 - 1) -
> kNumAlignBits) << kNumBitPriceShiftBits);
>
> + for (slot = kEndPosModelIndex / 2; slot < distTableSize2; slot++)
>
> {
>
> - distancesPrices[0] = posSlotPrices[0];
>
> - distancesPrices[1] = posSlotPrices[1];
>
> - distancesPrices[2] = posSlotPrices[2];
>
> - distancesPrices[3] = posSlotPrices[3];
>
> + posSlotPrices[(size_t)slot * 2 ] += delta;
>
> + posSlotPrices[(size_t)slot * 2 + 1] += delta;
>
> + delta += ((UInt32)1 << kNumBitPriceShiftBits);
>
> }
>
> + }
>
> +
>
> + {
>
> + UInt32 *dp = p->distancesPrices[lps];
>
> +
>
> + dp[0] = posSlotPrices[0];
>
> + dp[1] = posSlotPrices[1];
>
> + dp[2] = posSlotPrices[2];
>
> + dp[3] = posSlotPrices[3];
>
> +
>
> for (i = 4; i < kNumFullDistances; i += 2)
>
> {
>
> UInt32 slotPrice = posSlotPrices[GetPosSlot1(i)];
>
> - distancesPrices[i ] = slotPrice + tempPrices[i];
>
> - distancesPrices[i + 1] = slotPrice + tempPrices[i + 1];
>
> + dp[i ] = slotPrice + tempPrices[i];
>
> + dp[i + 1] = slotPrice + tempPrices[i + 1];
>
> }
>
> }
>
> }
>
> + // }
>
> }
>
>
>
>
>
> @@ -2245,10 +2410,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> if (len != 1)
>
> {
>
> LenEnc_Encode(&p->repLenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->repLenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->repLenEnc, posState,
> &p->repLenProbs, p->ProbPrices);
>
> -
>
> + --p->repLenEncCounter;
>
> p->state = kRepNextStates[p->state];
>
> }
>
> }
>
> @@ -2260,9 +2422,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> p->state = kMatchNextStates[p->state];
>
>
>
> LenEnc_Encode(&p->lenProbs, &p->rc, len -
> LZMA_MATCH_LEN_MIN, posState);
>
> - if (!p->fastMode)
>
> - if (--p->lenEnc.counters[posState] == 0)
>
> - LenPriceEnc_UpdateTable(&p->lenEnc, posState,
> &p->lenProbs, p->ProbPrices);
>
> + // --p->lenEnc.counter;
>
>
>
> dist -= LZMA_NUM_REPS;
>
> p->reps[3] = p->reps[2];
>
> @@ -2274,17 +2434,17 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> GetPosSlot(dist, posSlot);
>
> // RcTree_Encode_PosSlot(&p->rc,
> p->posSlotEncoder[GetLenToPosState(len)], posSlot);
>
> {
>
> - UInt32 symbol = posSlot + (1 << kNumPosSlotBits);
>
> + UInt32 sym = (UInt32)posSlot + (1 << kNumPosSlotBits);
>
> range = p->rc.range;
>
> probs = p->posSlotEncoder[GetLenToPosState(len)];
>
> do
>
> {
>
> - CLzmaProb *prob = probs + (symbol >> kNumPosSlotBits);
>
> - UInt32 bit = (symbol >> (kNumPosSlotBits - 1)) & 1;
>
> - symbol <<= 1;
>
> + CLzmaProb *prob = probs + (sym >> kNumPosSlotBits);
>
> + UInt32 bit = (sym >> (kNumPosSlotBits - 1)) & 1;
>
> + sym <<= 1;
>
> RC_BIT(&p->rc, prob, bit);
>
> }
>
> - while (symbol < (1 << kNumPosSlotBits * 2));
>
> + while (sym < (1 << kNumPosSlotBits * 2));
>
> p->rc.range = range;
>
> }
>
>
>
> @@ -2295,7 +2455,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p,
> UInt32 maxPackSize, UInt32 maxUnpa
> if (dist < kNumFullDistances)
>
> {
>
> unsigned base = ((2 | (posSlot & 1)) << footerBits);
>
> - RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, dist - base);
>
> + RcTree_ReverseEncode(&p->rc, p->posEncoders + base,
> footerBits, (unsigned)(dist /* - base */));
>
> }
>
> else
>
> {
>
> @@ -2331,14 +2491,14 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
> bit = dist & 1; dist >>= 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit); m = (m << 1) + bit;
>
> bit = dist & 1; RC_BIT(&p->rc,
> p->posAlignEncoder + m, bit);
>
> p->rc.range = range;
>
> - p->alignPriceCount++;
>
> + // p->alignPriceCount++;
>
> }
>
> }
>
> }
>
> }
>
> }
>
>
>
> - nowPos32 += len;
>
> + nowPos32 += (UInt32)len;
>
> p->additionalOffset -= len;
>
>
>
> if (p->additionalOffset == 0)
>
> @@ -2347,10 +2507,27 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc
> *p, UInt32 maxPackSize, UInt32 maxUnpa
>
>
> if (!p->fastMode)
>
> {
>
> - if (p->matchPriceCount >= (1 << 7))
>
> + /*
>
> + if (p->alignPriceCount >= 16) // kAlignTableSize
>
> + FillAlignPrices(p);
>
> + if (p->matchPriceCount >= 128)
>
> FillDistancesPrices(p);
>
> - if (p->alignPriceCount >= kAlignTableSize)
>
> + if (p->lenEnc.counter <= 0)
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + */
>
> + if (p->matchPriceCount >= 64)
>
> + {
>
> FillAlignPrices(p);
>
> + // { int y; for (y = 0; y < 100; y++) {
>
> + FillDistancesPrices(p);
>
> + // }}
>
> + LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb,
> &p->lenProbs, p->ProbPrices);
>
> + }
>
> + if (p->repLenEncCounter <= 0)
>
> + {
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> + LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb,
> &p->repLenProbs, p->ProbPrices);
>
> + }
>
> }
>
>
>
> if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
>
> @@ -2490,12 +2667,19 @@ void LzmaEnc_Init(CLzmaEnc *p)
>
>
> p->optEnd = 0;
>
> p->optCur = 0;
>
> +
>
> + {
>
> + for (i = 0; i < kNumOpts; i++)
>
> + p->opt[i].price = kInfinityPrice;
>
> + }
>
> +
>
> p->additionalOffset = 0;
>
>
>
> p->pbMask = (1 << p->pb) - 1;
>
> p->lpMask = ((UInt32)0x100 << p->lp) - ((unsigned)0x100 >> p->lc);
>
> }
>
>
>
> +
>
> void LzmaEnc_InitPrices(CLzmaEnc *p)
>
> {
>
> if (!p->fastMode)
>
> @@ -2507,6 +2691,9 @@ void LzmaEnc_InitPrices(CLzmaEnc *p)
> p->lenEnc.tableSize =
>
> p->repLenEnc.tableSize =
>
> p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
>
> +
>
> + p->repLenEncCounter = REP_LEN_COUNT;
>
> +
>
> LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, &p->lenProbs,
> p->ProbPrices);
>
> LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, &p->repLenProbs,
> p->ProbPrices);
>
> }
>
> @@ -2583,7 +2770,7 @@ typedef struct
> ISeqOutStream vt;
>
> Byte *data;
>
> SizeT rem;
>
> - Bool overflow;
>
> + BoolInt overflow;
>
> } CLzmaEnc_SeqOutStreamBuf;
>
>
>
> static size_t SeqOutStreamBuf_Write(const ISeqOutStream *pp, const void
> *data, size_t size)
>
> @@ -2615,7 +2802,7 @@ const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle
> pp)
> }
>
>
>
>
>
> -SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
>
> +SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, BoolInt reInit,
>
> Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32
> *unpackSize)
>
> {
>
> CLzmaEnc *p = (CLzmaEnc *)pp;
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> index d7426d38a4..04f9d98e7d 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt
> @@ -1,6 +1,28 @@
> HISTORY of the LZMA SDK
>
> -----------------------
>
>
>
> +19.00 2019-02-21
>
> +-------------------------
>
> +- Encryption strength for 7z archives was increased:
>
> + the size of random initialization vector was increased from 64-bit to
> 128-bit,
>
> + and the pseudo-random number generator was improved.
>
> +- The bug in 7zIn.c code was fixed.
>
> +
>
> +
>
> +18.06 2018-12-30
>
> +-------------------------
>
> +- The speed for LZMA/LZMA2 compressing was increased by 3-10%,
>
> + and there are minor changes in compression ratio.
>
> +- Some bugs were fixed.
>
> +- The bug in 7-Zip 18.02-18.05 was fixed:
>
> + There was memory leak in multithreading xz decoder -
> XzDecMt_Decode(),
>
> + if xz stream contains only one block.
>
> +- The changes for MSVS compiler makefiles:
>
> + - the makefiles now use "PLATFORM" macroname with values (x64, x86,
> arm64)
>
> + instead of "CPU" macroname with values (AMD64, ARM64).
>
> + - the makefiles by default now use static version of the run-time
library.
>
> +
>
> +
>
> 18.05 2018-04-30
>
> -------------------------
>
> - The speed for LZMA/LZMA2 compressing was increased
>
> diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> index a3deb20943..e17cd4fe1a 100644
> --- a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt
> @@ -1,4 +1,4 @@
> -LZMA SDK 18.05
>
> +LZMA SDK 19.00
>
> --------------
>
>
>
> LZMA SDK provides the documentation, samples, header files,
>
> --
> 2.27.0.windows.1
[-- Attachment #2: Type: text/html, Size: 384863 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-01-08 13:54 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-01-05 6:14 [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version Liu, WeiX C
2021-01-05 6:14 ` [PATCH 2/2] MdeModulePkg Lzma: Update LZMA SDK version to 19.00 Liu, WeiX C
2021-01-08 13:54 ` 回复: [edk2-devel] [PATCH 1/2] BaseTools LzmaCompress: Update LZMA to new 19.00 version gaoliming
-- strict thread matches above, loose matches on Subject: below --
2020-12-29 7:37 Liu, WeiX C
2020-12-15 2:06 Wei Liu
2020-12-17 1:30 ` 回复: " gaoliming
2020-12-20 13:33 ` Liu, WeiX C
2020-12-21 1:10 ` 回复: " gaoliming
2020-12-21 7:26 ` weix.c.liu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox