Initial commit of font compression code into public project.
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..e74c256
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,27 @@
+// Copyright (c) 2011 Google Inc. All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
+// met:
+//
+// * Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
+// * Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
+// * Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/README b/README
new file mode 100644
index 0000000..88dd8fa
--- /dev/null
+++ b/README
@@ -0,0 +1,155 @@
+This is a README for the font compression reference code. It’s very rough in
+this snapshot, but will be cleaned up some for public release.
+
+= How to run the compression test tool =
+
+This document documents how to run the compression reference code. At this
+writing, the code, while it is intended to produce a bytestream that can be
+reconstructed into a working font, the reference decompression code is not
+done, and the exact format of that bytestream is subject to change.
+
+== Building the tool ==
+
+On a standard Unix-style environment, it should be as simple as running “ant”.
+A couple of paths to compression subprocesses are hardcoded in
+CompressionRunner.java, namely “usr/bin/lzma” and “/bin/bzip2”. These are the
+default locations in Ubuntu, but if they’re elsewhere on your system, you’ll
+need to change that.
+
+The tool depends on sfntly for much of the font work. The lib/ directory
+contains a snapshot jar. If you want to use the latest sfntly sources, then cd
+to the java subdirectory, run “ant”, then copy these files dist/lib/sfntly.jar
+dist/tools/conversion/eot/eotconverter.jar and
+dist.tools/conversion/woff/woffconverter.jar to $(thisproject)/lib:
+
+dist/lib/sfntly.jar dist/tools/conversion/eot/eotconverter.jar
+dist.tools/conversion/woff/woffconverter.jar
+
+There’s also a dependency on guava (see references below).
+
+The dependencies are subject to their own licenses.
+
+== Setting up the test ==
+
+A run of the tool evaluates a “base” configuration plus one or more test
+configurations, for each font. It measures the file size of the test as a ratio
+over the base file size, then graphs the value of that ratio sorted across all
+files given on the command line.
+
+The test parameters are set by command line options (an improvement from the
+last snapshot). The base is set by the -b command line option, and the
+additional tests are specified by repeated -x command line options (see below).
+
+Each test is specified by a string description. It is a colon-separated list of
+stages. The final stage is entropy compression and can be one of “gzip”,
+“lzma”, “bzip2”, “woff”, “eot” (with actual wire-format MTX compression), or
+“uncomp” (for raw, uncompressed TTF’s). Also, the new wire-format draft
+WOFF2 spec is available as "woff2", and takes an entropy coding as an
+optional argument, as in "woff2/gzip" or "woff2/lzma".
+
+Other stages may optionally include subparameters (following a slash, and
+comma-separated). The stages are:
+
+glyf: performs glyf-table preprocessing based on MTX. There are subparameters:
+1. cbbox (composite bounding box). When specified, the bounding box for
+composite glyphs is included, otherwise stripped 2. sbbox (simple bounding
+box). When specified, the bounding box for simple glyphs is included 3. code:
+the bytecode is separated out into a separate stream 4. triplet: triplet coding
+(as in MTX) is used 5. push: push sequences are separated; if unset, pushes are
+kept inline in the bytecode 6. reslice: components of the glyf table are
+separated into individual streams, taking the MTX idea of separating the
+bytecodes further.
+
+hmtx: strips lsb’s from the hmtx table. Based on the idea that lsb’s can be
+reconstructed from bbox.
+
+hdmx: performs the delta coding on hdmx, essentially the same as MTX.
+
+cmap: compresses cmap table: wire format representation is inverse of cmap
+table plus exceptions (one glyph encoded by multiple character codes).
+
+kern: compresses kern table (not robust, intended just for rough testing).
+
+strip: the subparameters are a list of tables to be stripped entirely
+(comma-separated).
+
+The string roughly corresponding to MTX is:
+
+glyf/cbbox,code,triplet,push,hop:hdmx:gzip
+
+Meaning: glyph encoding is used, with simple glyph bboxes stripped (but
+composite glyph bboxes included), triplet coding, push sequences, and hop
+codes. The hdmx table is compressed. And finally, gzip is used as the entropy
+coder.
+
+This differs from MTX in a number of small ways: LZCOMP is not exactly the same
+as gzip. MTX uses three separate compression streams (the base font including
+triplet-coded glyph data), the bytecodes, and the push sequences, while this
+test uses a single stream. MTX also compresses the CVT table (an upper bound on
+the impact of this can be estimated by testing strip/cvt)
+
+Lastly, as a point of methodology, the code by default strips the “dsig” table,
+which would be invalidated by any non-bit-identical change to the font data. If
+it is desired to keep this table, add the “keepdsig” stage.
+
+The string representing the currently most aggressive optimization level is:
+
+glyf/triplet,code,push,reslice:hdmx:hmtx:cmap:kern:lzma
+
+In addition to the MTX one above, it strips the bboxes from composite glyphs,
+reslices the glyf table, compresses the htmx, cmap, and kern tables, and uses
+lzma as the entropy coding.
+
+The string corresponding to the current WOFF Ultra Condensed draft spec
+document is:
+
+glyf/cbbox,triplet,code,reslice:woff2/lzma
+
+The current C++ codebase can roundtrip compressed files as long as no per-table
+entropy coding is specified, as below (this will be fixed soon).
+
+glyf/cbbox,triplet,code,reslice:woff2
+
+
+== Running the tool ==
+
+java -jar build/jar/compression.jar *.ttf > chart.html
+
+The tool takes a list of OpenType fonts on the commandline, and generates an
+HTML chart, which it simply outputs to stdout. This chart uses the Google Chart
+API for plotting.
+
+Options:
+
+-b <desc>
+
+Sets the baseline experiment description.
+
+[ -x <desc> ]...
+
+Sets an experiment description. Can be used multiple times.
+
+-o
+
+Outputs the actual compressed file, substituting ".wof2" for ".ttf" in
+the input file name. Only useful when a single -x parameter is specified.
+
+= Decompressing the fonts =
+
+See the cpp/ directory (including cpp/README) for the C++ implementation of
+decompression. This code is based on OTS, and successfully roundtrips the
+basic compression as described in the draft spec.
+
+= References =
+
+sfntly: http://code.google.com/p/sfntly/ Guava:
+http://code.google.com/p/guava-libraries/ MTX:
+http://www.w3.org/Submission/MTX/
+
+Also please refer to documents (currently Google Docs):
+
+WOFF Ultra Condensed file format: proposals and discussion of wire format
+issues
+
+WIFF Ultra Condensed: more discussion of results and compression techniques.
+This tool was used to prepare the data in that document.
diff --git a/build.xml b/build.xml
new file mode 100644
index 0000000..8d48995
--- /dev/null
+++ b/build.xml
@@ -0,0 +1,31 @@
+<project name="compression" default="jar">
+ <target name="clean">
+ <delete dir="build/classes" />
+ <delete dir="build/jar" />
+ </target>
+
+ <target name="compile">
+ <mkdir dir="build/classes" />
+ <javac srcdir="src" destdir="build/classes" includeantruntime="false">
+ <compilerarg value="-Xlint" />
+ <classpath>
+ <fileset dir="lib" includes="*.jar" />
+ </classpath>
+ </javac>
+ </target>
+
+ <target name="jar" depends="compile">
+ <mkdir dir="build/jar" />
+ <jar destfile="build/jar/compression.jar" basedir="build/classes">
+ <zipfileset src="lib/eotconverter.jar" />
+ <zipfileset src="lib/guava-11.0.1.jar" />
+ <zipfileset src="lib/sfntly.jar" />
+ <zipfileset src="lib/woffconverter.jar" />
+ <manifest>
+ <attribute name="Main-Class" value="com.google.typography.font.compression.CompressionRunner" />
+ </manifest>
+ </jar>
+ </target>
+
+</project>
+
diff --git a/build/classes/com/google/typography/font/compression/AdvWidth.class b/build/classes/com/google/typography/font/compression/AdvWidth.class
new file mode 100644
index 0000000..80c4c03
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/AdvWidth.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/CmapEncoder.class b/build/classes/com/google/typography/font/compression/CmapEncoder.class
new file mode 100644
index 0000000..4aced8e
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/CmapEncoder.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/Command$StreamConsumer.class b/build/classes/com/google/typography/font/compression/Command$StreamConsumer.class
new file mode 100644
index 0000000..e6c24fb
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/Command$StreamConsumer.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/Command.class b/build/classes/com/google/typography/font/compression/Command.class
new file mode 100644
index 0000000..dd609f4
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/Command.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/CommandException.class b/build/classes/com/google/typography/font/compression/CommandException.class
new file mode 100644
index 0000000..3fe78f6
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/CommandException.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/CommandResult.class b/build/classes/com/google/typography/font/compression/CommandResult.class
new file mode 100644
index 0000000..eb0387d
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/CommandResult.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/CompressLzma.class b/build/classes/com/google/typography/font/compression/CompressLzma.class
new file mode 100644
index 0000000..5ee9e25
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/CompressLzma.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/CompressionRunner.class b/build/classes/com/google/typography/font/compression/CompressionRunner.class
new file mode 100644
index 0000000..6d08ce8
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/CompressionRunner.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/GlyfEncoder.class b/build/classes/com/google/typography/font/compression/GlyfEncoder.class
new file mode 100644
index 0000000..51a003f
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/GlyfEncoder.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/GzipUtil.class b/build/classes/com/google/typography/font/compression/GzipUtil.class
new file mode 100644
index 0000000..b63b757
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/GzipUtil.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/KernEncoder.class b/build/classes/com/google/typography/font/compression/KernEncoder.class
new file mode 100644
index 0000000..ca4bf20
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/KernEncoder.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/StatsCollector.class b/build/classes/com/google/typography/font/compression/StatsCollector.class
new file mode 100644
index 0000000..69f6cce
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/StatsCollector.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/TestCommand.class b/build/classes/com/google/typography/font/compression/TestCommand.class
new file mode 100644
index 0000000..402eda0
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/TestCommand.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/Woff2Writer$1.class b/build/classes/com/google/typography/font/compression/Woff2Writer$1.class
new file mode 100644
index 0000000..be891b2
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/Woff2Writer$1.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/Woff2Writer$CompressionType.class b/build/classes/com/google/typography/font/compression/Woff2Writer$CompressionType.class
new file mode 100644
index 0000000..6addfb9
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/Woff2Writer$CompressionType.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/Woff2Writer$TableDirectoryEntry.class b/build/classes/com/google/typography/font/compression/Woff2Writer$TableDirectoryEntry.class
new file mode 100644
index 0000000..7c80eca
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/Woff2Writer$TableDirectoryEntry.class
Binary files differ
diff --git a/build/classes/com/google/typography/font/compression/Woff2Writer.class b/build/classes/com/google/typography/font/compression/Woff2Writer.class
new file mode 100644
index 0000000..6e6f217
--- /dev/null
+++ b/build/classes/com/google/typography/font/compression/Woff2Writer.class
Binary files differ
diff --git a/build/jar/compression.jar b/build/jar/compression.jar
new file mode 100644
index 0000000..9e2b3ca
--- /dev/null
+++ b/build/jar/compression.jar
Binary files differ
diff --git a/cpp/.SConstruct.swp b/cpp/.SConstruct.swp
new file mode 100644
index 0000000..402a853
--- /dev/null
+++ b/cpp/.SConstruct.swp
Binary files differ
diff --git a/cpp/Inconsolata-Regular.woff2 b/cpp/Inconsolata-Regular.woff2
new file mode 100644
index 0000000..ddfefb3
--- /dev/null
+++ b/cpp/Inconsolata-Regular.woff2
Binary files differ
diff --git a/cpp/README b/cpp/README
new file mode 100644
index 0000000..6c40119
--- /dev/null
+++ b/cpp/README
@@ -0,0 +1,44 @@
+To build:
+
+Check out a recent version of OTS, so that ots-read-only/ is alongside cpp/
+
+Cd into cpp, and run "scons".
+
+Then, decompress a font using ./woff2-decompress font.wof2 > font.ttf .
+
+An example font is provided (Inconsolata-Regular.wof2). In this snapshot,
+it contains no actual compression, but does apply the glyf table transform
+and has the file format and structure as described in the current draft
+of the "WOFF Ultra Condensed file format" doc.
+
+That said, it is possible to get reliable estimates, and at the very least,
+bounds, on the compression efficiency, with the confidence that the
+compression is reversible. Running the compressor and doing a whole-file
+compression with a standard entropy coder such as gzip or lzma will yield
+a file size within a few dozen bytes or so of using a single entropy coded
+stream in the final file format.
+
+Another limitation of the current implementation snapshot is that it
+doesn't implement continue streams. These will follow shortly.
+
+= Building with lzma =
+
+The lzma-enabled build is made with gyp instead of scons. Right now, the build
+requires patching a clean copy of OTS. (And thanks to Bashi for the patch!)
+
+- Download GYP from http://code.google.com/p/gyp/
+- Download clean OTS sources:
+% svn checkout http://ots.googlecode.com/svn/trunk/ ots-read-only
+- Apply patch
+% cd ots-read-only; patch -p0 < ../ots-lzma.patch
+- Run gyp to generate Makefile
+% cd ../cpp; gyp --depth=. -f make woff2.gyp
+- Build
+% make
+
+Now run:
+
+out/Default/woff2-decompress Inconsolata-Regular-lzma.wof2 > i.ttf
+
+We expect the build recipes to be cleaned up before the code is ready for
+production, but this should be good enough for testing.
diff --git a/cpp/SConstruct b/cpp/SConstruct
new file mode 100644
index 0000000..4ec2e45
--- /dev/null
+++ b/cpp/SConstruct
@@ -0,0 +1,52 @@
+# Build script for Linux
+# Note: this script DOESN'T include LZMA. Use the gyp-based build instead.
+#
+# Usage:
+# $ cd ots/test/
+# $ scons -c # clean
+# $ scons # build
+#
+
+# Since the validator-checker tool might handle malicious font files, all hardening options for recent g++/ld are enabled just in case.
+# See http://wiki.debian.org/Hardening for details.
+env = Environment(CCFLAGS = ['-O2', '-I../ots-read-only/include', '-I../ots-read-only/src', '-I/usr/include/freetype2', '-ggdb', '-Wall', '-W', '-Wno-unused-parameter', '-fno-strict-aliasing', '-fPIE', '-fstack-protector', '-D_FORTIFY_SOURCE=2', '-DOTS_DEBUG'], LINKFLAGS = ['-ggdb', '-Wl,-z,relro', '-Wl,-z,now', '-pie', '-lz'])
+
+env.Library('libwoff2.a',
+ [
+ 'woff2.cc',
+
+ # Just build all of OTS to keep things simple for now. We could
+ # refactor so that we only compile the few support routines from
+ # ots.cc that we need.
+ '../ots-read-only/src/cff.cc',
+ '../ots-read-only/src/cff_type2_charstring.cc',
+ '../ots-read-only/src/cmap.cc',
+ '../ots-read-only/src/cvt.cc',
+ '../ots-read-only/src/fpgm.cc',
+ '../ots-read-only/src/gasp.cc',
+ '../ots-read-only/src/gdef.cc',
+ '../ots-read-only/src/glyf.cc',
+ '../ots-read-only/src/gpos.cc',
+ '../ots-read-only/src/gsub.cc',
+ '../ots-read-only/src/hdmx.cc',
+ '../ots-read-only/src/head.cc',
+ '../ots-read-only/src/hhea.cc',
+ '../ots-read-only/src/hmtx.cc',
+ '../ots-read-only/src/kern.cc',
+ '../ots-read-only/src/layout.cc',
+ '../ots-read-only/src/loca.cc',
+ '../ots-read-only/src/ltsh.cc',
+ '../ots-read-only/src/maxp.cc',
+ '../ots-read-only/src/metrics.cc',
+ '../ots-read-only/src/name.cc',
+ '../ots-read-only/src/os2.cc',
+ '../ots-read-only/src/ots.cc',
+ '../ots-read-only/src/post.cc',
+ '../ots-read-only/src/prep.cc',
+ '../ots-read-only/src/vdmx.cc',
+ '../ots-read-only/src/vhea.cc',
+ '../ots-read-only/src/vmtx.cc',
+ '../ots-read-only/src/vorg.cc'
+ ])
+
+env.Program('woff2-decompress.cc', LIBS = ['woff2'], LIBPATH='.')
diff --git a/cpp/woff2-decompress.cc b/cpp/woff2-decompress.cc
new file mode 100644
index 0000000..dc9a281
--- /dev/null
+++ b/cpp/woff2-decompress.cc
@@ -0,0 +1,61 @@
+// Copyright (c) 2012 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// A very simple commandline tool for decompressing woff2 format files
+// (given as argc[1]), writing the decompressed version to stdout.
+
+#include <fcntl.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include <cstdio>
+#include <cstdlib>
+
+#include "opentype-sanitiser.h"
+#include "woff2.h"
+
+namespace {
+
+int Usage(const char *argv0) {
+ std::fprintf(stderr, "Usage: %s woff2_file > dest_ttf_file\n", argv0);
+ return 1;
+}
+
+} // namespace
+
+int main(int argc, char **argv) {
+ if (argc != 2) return Usage(argv[0]);
+ if (::isatty(1)) return Usage(argv[0]);
+
+ const int fd = ::open(argv[1], O_RDONLY);
+ if (fd < 0) {
+ ::perror("open");
+ return 1;
+ }
+
+ struct stat st;
+ ::fstat(fd, &st);
+
+ uint8_t *data = new uint8_t[st.st_size];
+ if (::read(fd, data, st.st_size) != st.st_size) {
+ ::perror("read");
+ return 1;
+ }
+ ::close(fd);
+
+ size_t decompressed_size = ots::ComputeWOFF2FinalSize(data, st.st_size);
+ if (decompressed_size == 0) {
+ std::fprintf(stderr, "Error computing decompressed file size!\n");
+ return 1;
+ }
+ uint8_t *buf = new uint8_t[decompressed_size];
+ const bool result = ots::ConvertWOFF2ToTTF(buf, decompressed_size,
+ data, st.st_size);
+
+ if (!result) {
+ std::fprintf(stderr, "Failed to decompress file!\n");
+ }
+ fwrite(buf, 1, decompressed_size, stdout);
+ return !result;
+}
diff --git a/cpp/woff2.cc b/cpp/woff2.cc
new file mode 100644
index 0000000..902f4dd
--- /dev/null
+++ b/cpp/woff2.cc
@@ -0,0 +1,948 @@
+// Copyright (c) 2012 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// This is the implementation of decompression of the proposed WOFF Ultra
+// Condensed file format.
+
+// For now, use of LZMA is conditional, because the build is trickier. When
+// that gets all sorted out, we can get rid of these ifdefs.
+#define USE_LZMA
+
+#ifdef USE_LZMA
+#include "third_party/lzma_sdk/LzmaLib.h"
+#endif
+
+#include <zlib.h>
+#include <vector>
+#include "opentype-sanitiser.h"
+#include "ots-memory-stream.h"
+
+#include "ots.h"
+#include "woff2.h"
+
+namespace {
+
+// simple glyph flags
+const int kGlyfOnCurve = 1 << 0;
+const int kGlyfXShort = 1 << 1;
+const int kGlyfYShort = 1 << 2;
+const int kGlyfRepeat = 1 << 3;
+const int kGlyfThisXIsSame = 1 << 4;
+const int kGlyfThisYIsSame = 1 << 5;
+
+// composite glyph flags
+const int FLAG_ARG_1_AND_2_ARE_WORDS = 1 << 0;
+const int FLAG_ARGS_ARE_XY_VALUES = 1 << 1;
+const int FLAG_ROUND_XY_TO_GRID = 1 << 2;
+const int FLAG_WE_HAVE_A_SCALE = 1 << 3;
+const int FLAG_RESERVED = 1 << 4;
+const int FLAG_MORE_COMPONENTS = 1 << 5;
+const int FLAG_WE_HAVE_AN_X_AND_Y_SCALE = 1 << 6;
+const int FLAG_WE_HAVE_A_TWO_BY_TWO = 1 << 7;
+const int FLAG_WE_HAVE_INSTRUCTIONS = 1 << 8;
+const int FLAG_USE_MY_METRICS = 1 << 9;
+const int FLAG_OVERLAP_COMPOUND = 1 << 10;
+const int FLAG_SCALED_COMPONENT_OFFSET = 1 << 11;
+const int FLAG_UNSCALED_COMPONENT_OFFSET = 1 << 12;
+
+const size_t kSfntHeaderSize = 12;
+const size_t kSfntEntrySize = 16;
+const size_t kCheckSumAdjustmentOffset = 8;
+
+const size_t kEndPtsOfContoursOffset = 10;
+const size_t kCompositeGlyphBegin = 10;
+
+// Note that the byte order is big-endian, not the same as ots.cc
+#define TAG(a, b, c, d) ((a << 24) | (b << 16) | (c << 8) | d)
+
+const unsigned int kWoff2FlagsContinueStream = 1 << 4;
+const unsigned int kWoff2FlagsTransform = 1 << 5;
+
+const size_t kWoff2HeaderSize = 44;
+const size_t kWoff2EntrySize = 20;
+
+const size_t kLzmaHeaderSize = 13;
+
+const uint32_t kCompressionTypeMask = 0xf;
+const uint32_t kCompressionTypeNone = 0;
+const uint32_t kCompressionTypeGzip = 1;
+const uint32_t kCompressionTypeLzma = 2;
+
+
+struct Point {
+ int x;
+ int y;
+ bool on_curve;
+};
+
+struct Table {
+ uint32_t tag;
+ uint32_t flags;
+ uint32_t src_offset;
+ uint32_t src_length;
+
+ uint32_t transform_length;
+
+ uint32_t dst_offset;
+ uint32_t dst_length;
+};
+
+// Based on section 6.1.1 of MicroType Express draft spec
+bool Read255UShort(ots::Buffer *buf, unsigned int *value) {
+ const int kWordCode = 253;
+ const int kOneMoreByteCode2 = 254;
+ const int kOneMoreByteCode1 = 255;
+ const int kLowestUCode = 253;
+ uint8_t code = 0;
+ if (!buf->ReadU8(&code)) {
+ return OTS_FAILURE();
+ }
+ if (code == kWordCode) {
+ uint16_t result = 0;
+ if (!buf->ReadU16(&result)) {
+ return OTS_FAILURE();
+ }
+ *value = result;
+ return true;
+ } else if (code == kOneMoreByteCode1) {
+ uint8_t result = 0;
+ if (!buf->ReadU8(&result)) {
+ return OTS_FAILURE();
+ }
+ *value = result + kLowestUCode;
+ return true;
+ } else if (code == kOneMoreByteCode2) {
+ uint8_t result = 0;
+ if (!buf->ReadU8(&result)) {
+ return OTS_FAILURE();
+ }
+ *value = result + kLowestUCode * 2;
+ return true;
+ } else {
+ *value = code;
+ return true;
+ }
+}
+
+bool ReadBase128(ots::Buffer *buf, uint32_t *value) {
+ uint32_t result = 0;
+ for (size_t i = 0; i < 5; ++i) {
+ uint8_t code = 0;
+ if (!buf->ReadU8(&code)) {
+ return OTS_FAILURE();
+ }
+ result = (result << 7) | (code & 0x7f);
+ if ((code & 0x80) == 0) {
+ *value = result;
+ return true;
+ }
+ }
+ // Make sure not to exceed the size bound
+ return OTS_FAILURE();
+}
+
+size_t StoreU32(uint8_t *dst, size_t offset, uint32_t x) {
+ dst[offset] = x >> 24;
+ dst[offset + 1] = (x >> 16) & 0xff;
+ dst[offset + 2] = (x >> 8) & 0xff;
+ dst[offset + 3] = x & 0xff;
+ return offset + 4;
+}
+
+size_t Store16(uint8_t *dst, size_t offset, int x) {
+ dst[offset] = x >> 8;
+ dst[offset + 1] = x & 0xff;
+ return offset + 2;
+}
+
+int WithSign(int flag, int baseval) {
+ return (flag & 1) ? baseval : -baseval;
+}
+
+bool TripletDecode(const uint8_t *flags_in, const uint8_t *in, size_t in_size,
+ unsigned int n_points, std::vector<Point> *result,
+ size_t *in_bytes_consumed) {
+ int x = 0;
+ int y = 0;
+
+ if (n_points > in_size) {
+ return false;
+ }
+ unsigned int triplet_index = 0;
+
+ for (unsigned int i = 0; i < n_points; ++i) {
+ uint8_t flag = flags_in[i];
+ bool on_curve = !(flag >> 7);
+ flag &= 0x7f;
+ int n_data_bytes;
+ if (flag < 84) {
+ n_data_bytes = 1;
+ } else if (flag < 120) {
+ n_data_bytes = 2;
+ } else if (flag < 124) {
+ n_data_bytes = 3;
+ } else {
+ n_data_bytes = 4;
+ }
+#if 0
+ fprintf(stderr, "flag = %d:", flag);
+ for (int j = 0; j < n_data_bytes; ++j) {
+ fprintf(stderr, " %d", in[triplet_index + j]);
+ }
+ fprintf(stderr, "\n");
+#endif
+ if (triplet_index + n_data_bytes > in_size ||
+ triplet_index + n_data_bytes < triplet_index) {
+ return OTS_FAILURE();
+ }
+ int dx, dy;
+ if (flag < 10) {
+ dx = 0;
+ dy = WithSign(flag, ((flag & 14) << 7) + in[triplet_index]);
+ } else if (flag < 20) {
+ dx = WithSign(flag, (((flag - 10) & 14) << 7) + in[triplet_index]);
+ dy = 0;
+ } else if (flag < 84) {
+ int b0 = flag - 20;
+ int b1 = in[triplet_index];
+ dx = WithSign(flag, 1 + (b0 & 0x30) + (b1 >> 4));
+ dy = WithSign(flag >> 1, 1 + ((b0 & 0x0c) << 2) + (b1 & 0x0f));
+ } else if (flag < 120) {
+ int b0 = flag - 84;
+ dx = WithSign(flag, 1 + ((b0 / 12) << 8) + in[triplet_index]);
+ dy = WithSign(flag >> 1,
+ 1 + (((b0 % 12) >> 2) << 8) + in[triplet_index + 1]);
+ } else if (flag < 124) {
+ int b2 = in[triplet_index + 1];
+ dx = WithSign(flag, (in[triplet_index] << 4) + (b2 >> 4));
+ dy = WithSign(flag >> 1, ((b2 & 0x0f) << 8) + in[triplet_index + 2]);
+ } else {
+ dx = WithSign(flag, (in[triplet_index] << 8) + in[triplet_index + 1]);
+ dy = WithSign(flag >> 1,
+ (in[triplet_index + 2] << 8) + in[triplet_index + 3]);
+ }
+ triplet_index += n_data_bytes;
+ x += dx;
+ y += dy;
+ result->push_back(Point());
+ Point &back = result->back();
+ back.x = x;
+ back.y = y;
+ back.on_curve = on_curve;
+ // fprintf(stderr, "point %d: %d %d %s (delta %d %d)\n",
+ // i, x, y, on_curve ? "on" : "off", dx, dy);
+ }
+ *in_bytes_consumed = triplet_index;
+ return true;
+}
+
+// This function stores just the point data. On entry, dst points to the
+// beginning of a simple glyph. Returns true on success.
+bool StorePoints(const std::vector<Point> &points,
+ unsigned int n_contours, unsigned int instruction_length,
+ uint8_t *dst, size_t dst_size, size_t *glyph_size) {
+ unsigned int flag_offset = kEndPtsOfContoursOffset + 2 * n_contours + 2 +
+ instruction_length;
+ int last_flag = -1;
+ int repeat_count = 0;
+ int last_x = 0;
+ int last_y = 0;
+ int x_bytes = 0;
+ int y_bytes = 0;
+
+ for (unsigned int i = 0; i < points.size(); ++i) {
+ const Point &point = points[i];
+ int flag = point.on_curve ? kGlyfOnCurve : 0;
+ int dx = point.x - last_x;
+ int dy = point.y - last_y;
+ if (dx == 0) {
+ flag |= kGlyfThisXIsSame;
+ } else if (dx > -256 && dx < 256) {
+ flag |= kGlyfXShort | (dx > 0 ? kGlyfThisXIsSame : 0);
+ x_bytes += 1;
+ } else {
+ x_bytes += 2;
+ }
+ if (dy == 0) {
+ flag |= kGlyfThisYIsSame;
+ } else if (dy > -256 && dy < 256) {
+ flag |= kGlyfYShort | (dy > 0 ? kGlyfThisYIsSame : 0);
+ y_bytes += 1;
+ } else {
+ y_bytes += 2;
+ }
+ // fprintf(stderr, "nominal flag = %d\n", flag);
+ if (flag == last_flag && repeat_count != 255) {
+ dst[flag_offset - 1] |= kGlyfRepeat;
+ repeat_count++;
+ } else {
+ if (repeat_count != 0) {
+ if (flag_offset >= dst_size) return OTS_FAILURE();
+ dst[flag_offset++] = repeat_count;
+ }
+ if (flag_offset >= dst_size) return OTS_FAILURE();
+ dst[flag_offset++] = flag;
+ repeat_count = 0;
+ }
+ last_x = point.x;
+ last_y = point.y;
+ last_flag = flag;
+ }
+
+ if (repeat_count != 0) {
+ if (flag_offset >= dst_size) return OTS_FAILURE();
+ dst[flag_offset++] = repeat_count;
+ }
+ if (flag_offset + x_bytes + y_bytes > dst_size ||
+ flag_offset + x_bytes + y_bytes < flag_offset) {
+ return OTS_FAILURE();
+ }
+
+ int x_offset = flag_offset;
+ int y_offset = flag_offset + x_bytes;
+ last_x = 0;
+ last_y = 0;
+ for (unsigned int i = 0; i < points.size(); ++i) {
+ int dx = points[i].x - last_x;
+ if (dx == 0) {
+ // pass
+ } else if (dx > -256 && dx < 256) {
+ dst[x_offset++] = std::abs(dx);
+ } else {
+ x_offset = Store16(dst, x_offset, dx);
+ }
+ last_x += dx;
+ int dy = points[i].y - last_y;
+ if (dy == 0) {
+ // pass
+ } else if (dy > -256 && dy < 256) {
+ dst[y_offset++] = std::abs(dy);
+ } else {
+ y_offset = Store16(dst, y_offset, dy);
+ }
+ last_y += dy;
+ }
+ *glyph_size = y_offset;
+ return true;
+}
+
+// Compute the bounding box of the coordinates, and store into a glyf buffer.
+// A precondition is that there are at least 10 bytes available.
+void ComputeBbox(const std::vector<Point> &points, uint8_t *dst) {
+ int x_min = 0;
+ int y_min = 0;
+ int x_max = 0;
+ int y_max = 0;
+
+ for (unsigned int i = 0; i < points.size(); ++i) {
+ int x = points[i].x;
+ int y = points[i].y;
+ if (i == 0 || x < x_min) x_min = x;
+ if (i == 0 || x > x_max) x_max = x;
+ if (i == 0 || y < y_min) y_min = y;
+ if (i == 0 || y > y_max) y_max = y;
+ }
+ size_t offset = 2;
+ offset = Store16(dst, offset, x_min);
+ offset = Store16(dst, offset, y_min);
+ offset = Store16(dst, offset, x_max);
+ offset = Store16(dst, offset, y_max);
+}
+
+// Process entire bbox stream. This is done as a separate pass to allow for
+// composite bbox computations (an optional more aggressive transform).
+bool ProcessBboxStream(ots::Buffer *bbox_stream, unsigned int n_glyphs,
+ const std::vector<uint32_t> &loca_values, uint8_t *glyf_buf) {
+ const uint8_t *buf = bbox_stream->buffer();
+ unsigned int bitmap_length = ((n_glyphs + 31) >> 5) << 2;
+ if (bbox_stream->length() < bitmap_length) {
+ return OTS_FAILURE();
+ }
+ bbox_stream->Skip(bitmap_length);
+ for (unsigned int i = 0; i < n_glyphs; ++i) {
+ if (buf[i >> 3] & (0x80 >> (i & 7))) {
+ uint32_t loca_offset = loca_values[i];
+ if (loca_values[i + 1] - loca_offset < kEndPtsOfContoursOffset) {
+ return OTS_FAILURE();
+ }
+ bbox_stream->Read(glyf_buf + loca_offset + 2, 8);
+ }
+ }
+ return true;
+}
+
+bool ProcessComposite(ots::Buffer *composite_stream, uint8_t *dst,
+ size_t dst_size, size_t *glyph_size, bool *have_instructions) {
+ size_t start_offset = composite_stream->offset();
+ bool we_have_instructions = false;
+
+ uint16_t flags = FLAG_MORE_COMPONENTS;
+ while (flags & FLAG_MORE_COMPONENTS) {
+ if (!composite_stream->ReadU16(&flags)) {
+ return OTS_FAILURE();
+ }
+ we_have_instructions |= (flags & FLAG_WE_HAVE_INSTRUCTIONS) != 0;
+ size_t arg_size = 2; // glyph index
+ if (flags & FLAG_ARG_1_AND_2_ARE_WORDS) {
+ arg_size += 4;
+ } else {
+ arg_size += 2;
+ }
+ if (flags & FLAG_WE_HAVE_A_SCALE) {
+ arg_size += 2;
+ } else if (flags & FLAG_WE_HAVE_AN_X_AND_Y_SCALE) {
+ arg_size += 4;
+ } else if (flags & FLAG_WE_HAVE_A_TWO_BY_TWO) {
+ arg_size += 8;
+ }
+ if (!composite_stream->Skip(arg_size)) {
+ return OTS_FAILURE();
+ }
+ //fprintf(stderr, "flags = %04x, arg_size = %d\n", flags, arg_size);
+ }
+ size_t composite_glyph_size = composite_stream->offset() - start_offset;
+ if (composite_glyph_size + kCompositeGlyphBegin > dst_size) {
+ return OTS_FAILURE();
+ }
+ Store16(dst, 0, 0xffff); // nContours = -1 for composite glyph
+ std::memcpy(dst + kCompositeGlyphBegin,
+ composite_stream->buffer() + start_offset,
+ composite_glyph_size);
+ *glyph_size = kCompositeGlyphBegin + composite_glyph_size;
+ *have_instructions = we_have_instructions;
+ return true;
+}
+
+// Build TrueType loca table
+bool StoreLoca(const std::vector<uint32_t> &loca_values, int index_format,
+ uint8_t *dst, size_t dst_size) {
+ size_t loca_size = loca_values.size();
+ size_t offset_size = index_format ? 4 : 2;
+ if (offset_size * loca_size > dst_size) {
+ return OTS_FAILURE();
+ }
+ size_t offset = 0;
+ for (size_t i = 0; i < loca_values.size(); ++i) {
+ int value = loca_values[i];
+ if (index_format) {
+ offset = StoreU32(dst, offset, value);
+ } else {
+ offset = Store16(dst, offset, value >> 1);
+ }
+ }
+ return true;
+}
+
+// Reconstruct entire glyf table based on transformed original
+bool ReconstructGlyf(const uint8_t *data, size_t data_size,
+ uint8_t *dst, size_t dst_size,
+ uint8_t *loca_buf, size_t loca_size) {
+ ots::Buffer file(data, data_size);
+ uint32_t version;
+ const int kNumSubStreams = 7;
+ std::vector<std::pair<const uint8_t*, size_t> > substreams(kNumSubStreams);
+
+ if (!file.ReadU32(&version)) {
+ return OTS_FAILURE();
+ }
+ uint16_t num_glyphs = 0;
+ uint16_t index_format = 0;
+ if (!file.ReadU16(&num_glyphs) ||
+ !file.ReadU16(&index_format)) {
+ return OTS_FAILURE();
+ }
+ // fprintf(stderr, "num_glyphs = %d\n", num_glyphs);
+ unsigned int offset = (2 + kNumSubStreams) * 4;
+ for (int i = 0; i < kNumSubStreams; ++i) {
+ uint32_t substream_size = 0;
+ if (!file.ReadU32(&substream_size)) {
+ return OTS_FAILURE();
+ }
+ if (substream_size > data_size - offset) {
+ return OTS_FAILURE();
+ }
+ // fprintf(stderr, "substream size = %d\n", substream_size);
+ substreams[i] = std::make_pair(data + offset, substream_size);
+ offset += substream_size;
+ }
+ ots::Buffer n_contour_stream(substreams[0].first, substreams[0].second);
+ ots::Buffer n_points_stream(substreams[1].first, substreams[1].second);
+ ots::Buffer flag_stream(substreams[2].first, substreams[2].second);
+ ots::Buffer glyph_stream(substreams[3].first, substreams[3].second);
+ ots::Buffer composite_stream(substreams[4].first, substreams[4].second);
+ ots::Buffer bbox_stream(substreams[5].first, substreams[5].second);
+ ots::Buffer instruction_stream(substreams[6].first, substreams[6].second);
+
+ std::vector<uint32_t> loca_values(num_glyphs + 1);
+ std::vector<unsigned int> n_points_vec;
+ std::vector<Point> points;
+ uint32_t loca_offset = 0;
+ for (unsigned int i = 0; i < num_glyphs; ++i) {
+ size_t glyph_size = 0;
+ uint16_t n_contours = 0;
+ if (!n_contour_stream.ReadU16(&n_contours)) {
+ return OTS_FAILURE();
+ }
+ // fprintf(stderr, "n_contours[%d] = %d\n", i, n_contours);
+ uint8_t *glyf_dst = dst + loca_offset;
+ size_t glyf_dst_size = dst_size - loca_offset;
+ if (n_contours == 0xffff) {
+ // composite glyph
+ bool have_instructions = false;
+ unsigned int instruction_size = 0;
+ if (!ProcessComposite(&composite_stream, glyf_dst, glyf_dst_size,
+ &glyph_size, &have_instructions)) {
+ return OTS_FAILURE();
+ }
+ if (have_instructions) {
+ if (!Read255UShort(&glyph_stream, &instruction_size)) {
+ return OTS_FAILURE();
+ }
+ if (instruction_size + 2 > glyf_dst_size - glyph_size) {
+ return OTS_FAILURE();
+ }
+ Store16(glyf_dst, glyph_size, instruction_size);
+ if (!instruction_stream.Read(glyf_dst + glyph_size + 2,
+ instruction_size)) {
+ return OTS_FAILURE();
+ }
+ glyph_size += instruction_size + 2;
+ }
+ } else if (n_contours > 0) {
+ // simple glyph
+ n_points_vec.clear();
+ points.clear();
+ unsigned int total_n_points = 0;
+ unsigned int n_points_contour;
+ for (unsigned int j = 0; j < n_contours; ++j) {
+ if (!Read255UShort(&n_points_stream, &n_points_contour)) {
+ return OTS_FAILURE();
+ }
+ n_points_vec.push_back(n_points_contour);
+ total_n_points += n_points_contour;
+ }
+ unsigned int flag_size = total_n_points;
+ if (flag_size > flag_stream.length() - flag_stream.offset()) {
+ return OTS_FAILURE();
+ }
+ const uint8_t *flags_buf = flag_stream.buffer() + flag_stream.offset();
+ const uint8_t *triplet_buf = glyph_stream.buffer() +
+ glyph_stream.offset();
+ size_t triplet_size = glyph_stream.length() - glyph_stream.offset();
+ size_t triplet_bytes_consumed = 0;
+ if (!TripletDecode(flags_buf, triplet_buf, triplet_size, total_n_points,
+ &points, &triplet_bytes_consumed)) {
+ return OTS_FAILURE();
+ }
+ if (glyf_dst_size < kEndPtsOfContoursOffset + 2 * n_contours) {
+ return OTS_FAILURE();
+ }
+ Store16(glyf_dst, 0, n_contours);
+ ComputeBbox(points, glyf_dst);
+ size_t offset = kEndPtsOfContoursOffset;
+ int end_point = -1;
+ for (unsigned int contour_ix = 0; contour_ix < n_contours; ++contour_ix) {
+ end_point += n_points_vec[contour_ix];
+ offset = Store16(glyf_dst, offset, end_point);
+ }
+ flag_stream.Skip(flag_size);
+ glyph_stream.Skip(triplet_bytes_consumed);
+ unsigned int instruction_size;
+ if (!Read255UShort(&glyph_stream, &instruction_size)) {
+ return OTS_FAILURE();
+ }
+ // fprintf(stderr, "%d: instruction size = %d\n", i, instruction_size);
+ uint8_t *instruction_dst = glyf_dst + kEndPtsOfContoursOffset +
+ 2 * n_contours;
+ Store16(instruction_dst, 0, instruction_size);
+ if (!instruction_stream.Read(instruction_dst + 2, instruction_size)) {
+ return OTS_FAILURE();
+ }
+ if (!StorePoints(points, n_contours, instruction_size,
+ glyf_dst, glyf_dst_size, &glyph_size)) {
+ return OTS_FAILURE();
+ }
+ } else {
+ glyph_size = 0;
+ }
+ loca_values[i] = loca_offset;
+ if (glyph_size + 3 < glyph_size) {
+ return OTS_FAILURE();
+ }
+ // Round up to 4-byte alignment
+ glyph_size = (glyph_size + 3) & -4;
+ if (glyph_size > dst_size - loca_offset) {
+ // This shouldn't happen, but this test defensively maintains the
+ // invariant that loca_offset <= dst_size.
+ return OTS_FAILURE();
+ }
+ loca_offset += glyph_size;
+ }
+ loca_values[num_glyphs] = loca_offset;
+ if (!ProcessBboxStream(&bbox_stream, num_glyphs, loca_values, dst)) {
+ return OTS_FAILURE();
+ }
+ return StoreLoca(loca_values, index_format, loca_buf, loca_size);
+}
+
+// This is linear search, but could be changed to binary because we
+// do have a guarantee that the tables are sorted by tag. But the total
+// cpu time is expected to be very small in any case.
+const Table *FindTable(const std::vector<Table> &tables, uint32_t tag) {
+ size_t n_tables = tables.size();
+ for (size_t i = 0; i < n_tables; ++i) {
+ if (tables[i].tag == tag) {
+ return &tables[i];
+ }
+ }
+ return NULL;
+}
+
+bool ReconstructTransformed(const std::vector<Table> &tables, uint32_t tag,
+ const uint8_t *transformed_buf, size_t transformed_size,
+ uint8_t *dst) {
+ if (tag == TAG('g', 'l', 'y', 'f')) {
+ const Table *glyf_table = FindTable(tables, tag);
+ const Table *loca_table = FindTable(tables, TAG('l', 'o', 'c', 'a'));
+ if (glyf_table == NULL || loca_table == NULL) {
+ return OTS_FAILURE();
+ }
+ return ReconstructGlyf(transformed_buf, transformed_size,
+ dst + glyf_table->dst_offset, glyf_table->dst_length,
+ dst + loca_table->dst_offset, loca_table->dst_offset);
+ } else if (tag == TAG('l', 'o', 'c', 'a')) {
+ // processing was already done by glyf table, but validate
+ if (!FindTable(tables, TAG('g', 'l', 'y', 'f'))) {
+ return OTS_FAILURE();
+ }
+ } else {
+ // transform for the tag is not known
+ return OTS_FAILURE();
+ }
+ return true;
+}
+
+// TODO: copied from ots.cc, probably shouldn't be duplicated.
+// Round a value up to the nearest multiple of 4. Don't round the value in the
+// case that rounding up overflows.
+template<typename T> T Round4(T value) {
+ if (std::numeric_limits<T>::max() - value < 3) {
+ return value;
+ }
+ return (value + 3) & ~3;
+}
+
+uint32_t ComputeChecksum(const uint8_t *buf, size_t size) {
+ uint32_t checksum = 0;
+ for (size_t i = 0; i < size; i += 4) {
+ // We assume the addition is mod 2^32. This is a pretty safe assumption,
+ // but technically it's undefined behavior.
+ checksum += (buf[i] << 24) | (buf[i + 1] << 16) |
+ (buf[i + 2] << 8) | buf[i + 3];
+ }
+ return checksum;
+}
+
+bool FixChecksums(const std::vector<Table> &tables, uint8_t *dst) {
+ const Table *head_table = FindTable(tables, TAG('h', 'e', 'a', 'd'));
+ if (head_table == NULL ||
+ head_table->dst_length < kCheckSumAdjustmentOffset + 4) {
+ return OTS_FAILURE();
+ }
+ size_t adjustment_offset = head_table->dst_offset + kCheckSumAdjustmentOffset;
+ StoreU32(dst, adjustment_offset, 0);
+ size_t n_tables = tables.size();
+ uint32_t file_checksum = 0;
+ for (size_t i = 0; i < n_tables; ++i) {
+ const Table *table = &tables[i];
+ size_t table_length = table->dst_length;
+ uint8_t *table_data = dst + table->dst_offset;
+ uint32_t checksum = ComputeChecksum(table_data, table_length);
+ StoreU32(dst, kSfntHeaderSize + i * kSfntEntrySize + 4, checksum);
+ file_checksum += checksum;
+ }
+ file_checksum += ComputeChecksum(dst,
+ kSfntHeaderSize + kSfntEntrySize * n_tables);
+ uint32_t checksum_adjustment = 0xb1b0afba - file_checksum;
+ StoreU32(dst, adjustment_offset, checksum_adjustment);
+ return true;
+}
+
+bool Woff2Uncompress(uint8_t *dst_buf, size_t dst_size,
+ const uint8_t *src_buf, size_t src_size, uint32_t compression_type) {
+ if (compression_type == kCompressionTypeGzip) {
+ uLongf uncompressed_length = dst_size;
+ int r = uncompress((Bytef *)dst_buf, &uncompressed_length,
+ src_buf, src_size);
+ if (r != Z_OK || uncompressed_length != src_size) {
+ return OTS_FAILURE();
+ }
+ return true;
+#ifdef USE_LZMA
+ } else if (compression_type == kCompressionTypeLzma) {
+ if (src_size < kLzmaHeaderSize) {
+ // Make sure we have at least a full Lzma header
+ return OTS_FAILURE();
+ }
+ // TODO: check that size matches (or elide size?)
+ size_t uncompressed_size = dst_size;
+ size_t compressed_size = src_size;
+ int result = LzmaUncompress(dst_buf, &dst_size,
+ src_buf + kLzmaHeaderSize, &compressed_size,
+ src_buf, LZMA_PROPS_SIZE);
+ if (result != SZ_OK || uncompressed_size != dst_size) {
+ return OTS_FAILURE();
+ }
+ return true;
+#endif
+ }
+ // Unknown compression type
+ return OTS_FAILURE();
+}
+
+bool ReadLongDirectory(ots::Buffer *file, std::vector<Table> *tables,
+ size_t num_tables) {
+ for (size_t i = 0; i < num_tables; ++i) {
+ Table *table = &(*tables)[i];
+ if (!file->ReadU32(&table->tag) ||
+ !file->ReadU32(&table->flags) ||
+ !file->ReadU32(&table->src_length) ||
+ !file->ReadU32(&table->transform_length) ||
+ !file->ReadU32(&table->dst_length)) {
+ return OTS_FAILURE();
+ }
+ }
+ return true;
+}
+
+const uint32_t known_tags[29] = {
+ TAG('c', 'm', 'a', 'p'), // 0
+ TAG('h', 'e', 'a', 'd'), // 1
+ TAG('h', 'h', 'e', 'a'), // 2
+ TAG('h', 'm', 't', 'x'), // 3
+ TAG('m', 'a', 'x', 'p'), // 4
+ TAG('n', 'a', 'm', 'e'), // 5
+ TAG('O', 'S', '/', '2'), // 6
+ TAG('p', 'o', 's', 't'), // 7
+ TAG('c', 'v', 't', ' '), // 8
+ TAG('f', 'p', 'g', 'm'), // 9
+ TAG('g', 'l', 'y', 'f'), // 10
+ TAG('l', 'o', 'c', 'a'), // 11
+ TAG('p', 'r', 'e', 'p'), // 12
+ TAG('C', 'F', 'F', ' '), // 13
+ TAG('V', 'O', 'R', 'G'), // 14
+ TAG('E', 'B', 'D', 'T'), // 15
+ TAG('E', 'B', 'L', 'C'), // 16
+ TAG('g', 'a', 's', 'p'), // 17
+ TAG('h', 'd', 'm', 'x'), // 18
+ TAG('k', 'e', 'r', 'n'), // 19
+ TAG('L', 'T', 'S', 'H'), // 20
+ TAG('P', 'C', 'L', 'T'), // 21
+ TAG('V', 'D', 'M', 'X'), // 22
+ TAG('v', 'h', 'e', 'a'), // 23
+ TAG('v', 'm', 't', 'x'), // 24
+ TAG('B', 'A', 'S', 'E'), // 25
+ TAG('G', 'D', 'E', 'F'), // 26
+ TAG('G', 'P', 'O', 'S'), // 27
+ TAG('G', 'S', 'U', 'B'), // 28
+};
+
+bool ReadShortDirectory(ots::Buffer *file, std::vector<Table> *tables,
+ size_t num_tables) {
+ uint32_t last_compression_type = 0;
+ for (size_t i = 0; i < num_tables; ++i) {
+ Table *table = &(*tables)[i];
+ uint8_t flag_byte = 0;
+ if (!file->ReadU8(&flag_byte)) {
+ return OTS_FAILURE();
+ }
+ uint32_t tag = 0;
+ if ((flag_byte & 0x1f) == 0x1f) {
+ if (!file->ReadU32(&tag)) {
+ return OTS_FAILURE();
+ }
+ } else {
+ tag = known_tags[flag_byte & 0x1f];
+ }
+ uint32_t flags = flag_byte >> 6;
+ if (flags == 3) {
+ flags = last_compression_type | kWoff2FlagsContinueStream;
+ } else {
+ last_compression_type = flags;
+ }
+ if ((flag_byte & 0x20) != 0) {
+ flags |= kWoff2FlagsTransform;
+ }
+ uint32_t dst_length = 0;
+ if (!ReadBase128(file, &dst_length)) {
+ return OTS_FAILURE();
+ }
+ uint32_t transform_length = dst_length;
+ if ((flags & kWoff2FlagsTransform) != 0) {
+ if (!ReadBase128(file, &transform_length)) {
+ return OTS_FAILURE();
+ }
+ }
+ uint32_t src_length = transform_length;
+ if ((flag_byte >> 6) == 1 | (flag_byte >> 6) == 2) {
+ if (!ReadBase128(file, &src_length)) {
+ return OTS_FAILURE();
+ }
+ }
+ table->tag = tag;
+ table->flags = flags;
+ table->src_length = src_length;
+ table->transform_length = transform_length;
+ table->dst_length = dst_length;
+ }
+ return true;
+}
+
+} // namespace
+
+namespace ots {
+
+size_t ComputeWOFF2FinalSize(const uint8_t *data, size_t length) {
+ ots::Buffer file(data, length);
+
+ file.Skip(16);
+ uint32_t total_length = 0;
+ if (!file.ReadU32(&total_length)) {
+ return OTS_FAILURE();
+ }
+ return total_length;
+}
+
+bool ConvertWOFF2ToTTF(uint8_t *result, size_t result_length,
+ const uint8_t *data, size_t length) {
+ ots::Buffer file(data, length);
+
+ uint32_t signature = 0;
+ uint32_t flavor = 0;
+ if (!file.ReadU32(&signature) || signature != 0x774f4632 ||
+ !file.ReadU32(&flavor)) {
+ return OTS_FAILURE();
+ }
+ file.Skip(4);
+ uint16_t num_tables = 0;
+ if (!file.ReadU16(&num_tables)) {
+ return OTS_FAILURE();
+ }
+ file.Skip(30);
+ std::vector<Table> tables(num_tables);
+ // Note: change below to ReadLongDirectory to enable long format.
+ if (!ReadShortDirectory(&file, &tables, num_tables)) {
+ return OTS_FAILURE();
+ }
+ size_t src_offset = file.offset();
+ size_t dst_offset = kSfntHeaderSize + kSfntEntrySize * num_tables;
+ size_t uncompressed_sum = 0;
+ for (int i = 0; i < num_tables; ++i) {
+ Table *table = &tables[i];
+ table->src_offset = src_offset;
+ if (src_offset + table->src_length < src_offset) {
+ return OTS_FAILURE();
+ }
+ src_offset += table->src_length;
+ src_offset = Round4(src_offset); // TODO: reconsider
+ table->dst_offset = dst_offset;
+ if (dst_offset + table->dst_length < dst_offset) {
+ return OTS_FAILURE();
+ }
+ dst_offset += table->dst_length;
+ dst_offset = Round4(dst_offset);
+ if ((table->flags & kCompressionTypeMask) != kCompressionTypeNone) {
+ if (uncompressed_sum + table->src_length < uncompressed_sum) {
+ return OTS_FAILURE();
+ }
+ uncompressed_sum += table->src_length;
+ }
+ }
+ // Enforce same 30M limit on uncompressed tables as OTS
+ if (uncompressed_sum > 30 * 1024 * 1024) {
+ return OTS_FAILURE();
+ }
+ if (dst_offset > result_length) {
+ return OTS_FAILURE();
+ }
+
+ // Start building the font
+ size_t offset = 0;
+ offset = StoreU32(result, offset, flavor);
+ offset = Store16(result, offset, num_tables);
+ unsigned max_pow2 = 0;
+ while (1u << (max_pow2 + 1) <= num_tables) {
+ max_pow2++;
+ }
+ const uint16_t output_search_range = (1u << max_pow2) << 4;
+ offset = Store16(result, offset, output_search_range);
+ offset = Store16(result, offset, max_pow2);
+ offset = Store16(result, offset, (num_tables << 4) - output_search_range);
+ for (int i = 0; i < num_tables; ++i) {
+ const Table *table = &tables[i];
+ offset = StoreU32(result, offset, table->tag);
+ offset = StoreU32(result, offset, 0); // checksum, to fill in later
+ offset = StoreU32(result, offset, table->dst_offset);
+ offset = StoreU32(result, offset, table->dst_length);
+ }
+ std::vector<uint8_t> uncompressed_buf;
+ bool continue_valid = false;
+ for (int i = 0; i < num_tables; ++i) {
+ const Table *table = &tables[i];
+ uint32_t flags = table->flags;
+ const uint8_t *src_buf = data + table->src_offset;
+ uint32_t compression_type = flags & kCompressionTypeMask;
+ const uint8_t *transform_buf = NULL;
+ size_t transform_length = table->transform_length;
+ if ((flags & kWoff2FlagsContinueStream) != 0) {
+ if (!continue_valid) {
+ return OTS_FAILURE();
+ }
+ } else if (compression_type == kCompressionTypeNone) {
+ if (transform_length != table->src_length) {
+ return OTS_FAILURE();
+ }
+ transform_buf = src_buf;
+ continue_valid = false;
+ } else if ((flags & kWoff2FlagsContinueStream) == 0) {
+ size_t total_size = transform_length;
+ for (int j = i + 1; j < num_tables; ++j) {
+ if ((tables[j].flags & kWoff2FlagsContinueStream) == 0) {
+ break;
+ }
+ if (total_size + tables[j].transform_length < total_size) {
+ return OTS_FAILURE();
+ }
+ total_size += tables[j].transform_length;
+ }
+ uncompressed_buf.resize(total_size);
+ if (!Woff2Uncompress(&uncompressed_buf[0], total_size,
+ src_buf, table->src_length, compression_type)) {
+ return OTS_FAILURE();
+ }
+ transform_buf = &uncompressed_buf[0];
+ continue_valid = true;
+ }
+
+ if ((flags & kWoff2FlagsTransform) == 0) {
+ if (transform_length != table->dst_length) {
+ return OTS_FAILURE();
+ }
+ std::memcpy(result + table->dst_offset, transform_buf,
+ transform_length);
+ } else {
+ if (!ReconstructTransformed(tables, table->tag,
+ transform_buf, transform_length, result)) {
+ return OTS_FAILURE();
+ }
+ }
+ if (continue_valid) {
+ transform_buf += transform_length;
+ }
+ }
+
+ return FixChecksums(tables, result);
+}
+
+} // namespace ots
diff --git a/cpp/woff2.gyp b/cpp/woff2.gyp
new file mode 100644
index 0000000..6fd3dc9
--- /dev/null
+++ b/cpp/woff2.gyp
@@ -0,0 +1,43 @@
+{
+ 'variables': {
+ 'ots_include_dirs': [
+ # This isn't particularly elegant, but it works
+ '../ots-read-only/include',
+ '../ots-read-only/src',
+ ],
+ },
+ 'target_defaults': {
+ 'defines': [
+ 'OTS_DEBUG',
+ ],
+ },
+ 'targets': [
+ {
+ 'target_name': 'woff2',
+ 'type': 'static_library',
+ 'sources': [
+ 'woff2.cc',
+ ],
+ 'include_dirs': [
+ '<@(ots_include_dirs)',
+ ],
+ 'dependencies': [
+ '../ots-read-only/ots-standalone.gyp:ots',
+ ],
+ },
+ {
+ 'target_name': 'woff2-decompress',
+ 'type': 'executable',
+ 'sources': [
+ 'woff2-decompress.cc',
+ ],
+ 'include_dirs': [
+ '<@(ots_include_dirs)',
+ ],
+ 'dependencies': [
+ 'woff2',
+ ],
+ },
+ ],
+}
+
diff --git a/cpp/woff2.h b/cpp/woff2.h
new file mode 100644
index 0000000..64ae6df
--- /dev/null
+++ b/cpp/woff2.h
@@ -0,0 +1,21 @@
+// Copyright (c) 2012 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef OTS_WOFF2_H_
+#define OTS_WOFF2_H_
+
+namespace ots {
+
+// Compute the size of the final uncompressed font, or 0 on error.
+size_t ComputeWOFF2FinalSize(const uint8_t *data, size_t length);
+
+// Decompresses the font into the target buffer. The result_length should
+// be the same as determined by ComputeFinalSize(). Returns true on successful
+// decompression.
+bool ConvertWOFF2ToTTF(uint8_t *result, size_t result_length,
+ const uint8_t *data, size_t length);
+
+}
+
+#endif // OTS_WOFF2_H_
diff --git a/docs/WOFFUltraCondensedfileformat.pdf b/docs/WOFFUltraCondensedfileformat.pdf
new file mode 100644
index 0000000..24c1e04
--- /dev/null
+++ b/docs/WOFFUltraCondensedfileformat.pdf
Binary files differ
diff --git a/lib/eotconverter.jar b/lib/eotconverter.jar
new file mode 100644
index 0000000..6543d70
--- /dev/null
+++ b/lib/eotconverter.jar
Binary files differ
diff --git a/lib/guava-11.0.1.jar b/lib/guava-11.0.1.jar
new file mode 100644
index 0000000..af4a383
--- /dev/null
+++ b/lib/guava-11.0.1.jar
Binary files differ
diff --git a/lib/sfntly.jar b/lib/sfntly.jar
new file mode 100644
index 0000000..415d5c5
--- /dev/null
+++ b/lib/sfntly.jar
Binary files differ
diff --git a/lib/woffconverter.jar b/lib/woffconverter.jar
new file mode 100644
index 0000000..1ac0133
--- /dev/null
+++ b/lib/woffconverter.jar
Binary files differ
diff --git a/ots-lzma.patch b/ots-lzma.patch
new file mode 100644
index 0000000..0cf22a5
--- /dev/null
+++ b/ots-lzma.patch
@@ -0,0 +1,5500 @@
+Index: test/lzma.cc
+===================================================================
+--- test/lzma.cc (revision 0)
++++ test/lzma.cc (revision 0)
+@@ -0,0 +1,139 @@
++// Copyright (c) 2012 The Chromium Authors. All rights reserved.
++// Use of this source code is governed by a BSD-style license that can be
++// found in the LICENSE file.
++
++#include <arpa/inet.h>
++#include <fcntl.h>
++#include <sys/stat.h>
++#include <unistd.h>
++
++#include <cstdio>
++#include <string>
++#include <vector>
++
++#include "opentype-sanitiser.h"
++#include "ots-memory-stream.h"
++
++#include "third_party/lzma_sdk/LzmaLib.h"
++
++namespace {
++
++static const size_t kCompressedLengthFieldSize = 4;
++
++int Usage(const char *argv0) {
++ std::fprintf(stderr, "Usage: %s (compress|decompress) filename\n", argv0);
++ return 1;
++}
++
++bool ReadFile(const char *file_name, std::vector<uint8_t>* data) {
++ const int fd = open(file_name, O_RDONLY);
++ if (fd < 0) {
++ return false;
++ }
++
++ struct stat st;
++ fstat(fd, &st);
++
++ data->resize(st.st_size);
++ if (read(fd, &(*data)[0], st.st_size) != st.st_size) {
++ close(fd);
++ return false;
++ }
++ close(fd);
++ return true;
++}
++
++bool Compress(std::vector<uint8_t>* input, std::vector<uint8_t>* output) {
++ size_t props_size = LZMA_PROPS_SIZE;
++ size_t out_len = input->size() * 2;
++ output->resize(out_len + props_size + kCompressedLengthFieldSize);
++
++ uint8_t* output_start = &(*output)[kCompressedLengthFieldSize];
++
++ int result = LzmaCompress(output_start + LZMA_PROPS_SIZE, &out_len,
++ &(*input)[0], input->size(),
++ output_start, &props_size,
++ -1, 0, -1, -1, -1, -1, 1);
++ if (props_size != LZMA_PROPS_SIZE || result != SZ_OK)
++ return false;
++
++ output->resize(props_size + out_len + kCompressedLengthFieldSize);
++ // Store the uncompressed length at the beginning of buffer.
++ uint32_t uncompressed_length = htonl(input->size());
++ memcpy(&(*output)[0], &uncompressed_length, kCompressedLengthFieldSize);
++ return true;
++}
++
++bool Decompress(std::vector<uint8_t>* input, std::vector<uint8_t>* output) {
++ if (input->size() < kCompressedLengthFieldSize + LZMA_PROPS_SIZE)
++ return false;
++
++ // Assume the uncompressed length is stored at the beginning of the buffer
++ // in network byte order.
++ uint32_t uncompressed_length = 0;
++ memcpy(&uncompressed_length, &(*input)[0], kCompressedLengthFieldSize);
++ uncompressed_length = ntohl(uncompressed_length);
++
++ output->resize(uncompressed_length);
++ uint8_t* input_start = &(*input)[kCompressedLengthFieldSize];
++ size_t in_len = input->size() - LZMA_PROPS_SIZE;
++ size_t out_len = output->size();
++ int result = LzmaUncompress(&(*output)[0], &out_len,
++ input_start + LZMA_PROPS_SIZE,
++ &in_len, input_start, LZMA_PROPS_SIZE);
++
++ return result == SZ_OK;
++}
++
++bool DumpResult(std::vector<uint8_t>* result, const std::string* file_name) {
++ int fd = open(file_name->c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0600);
++ if (fd < 0) {
++ perror("opening output file");
++ return false;
++ }
++ if (write(fd, &(*result)[0], result->size()) < 0) {
++ perror("writing output file");
++ close(fd);
++ return false;
++ }
++ close(fd);
++ return true;
++}
++
++} // namespace
++
++int main(int argc, char** argv) {
++ if (argc != 3) return Usage(argv[0]);
++
++ std::vector<uint8_t> in_data;
++ if (!ReadFile(argv[2], &in_data)) {
++ std::fprintf(stderr, "Failed to read file!\n");
++ return 1;
++ }
++
++ std::vector<uint8_t> out_data;
++ std::string file_name;
++ if (std::strncmp("compress", argv[1], 8) == 0) {
++ if (!Compress(&in_data, &out_data)) {
++ std::fprintf(stderr, "Failed to compress file.\n");
++ return 1;
++ }
++ file_name = "compressed.dat";
++ } else if (std::strncmp("decompress", argv[1], 10) == 0) {
++ if (!Decompress(&in_data, &out_data)) {
++ std::fprintf(stderr, "Failed to decompress file.\n");
++ return 1;
++ }
++ file_name = "decompressed.dat";
++ } else {
++ std::fprintf(
++ stderr,
++ "The second argument must be either 'compress' or 'decompress'.");
++ return 1;
++ }
++
++ if (!DumpResult(&out_data, &file_name)) {
++ std::fprintf(stderr, "Failed to write the result.\n");
++ return 1;
++ }
++}
+
+Property changes on: test/lzma.cc
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: ots-common.gypi
+===================================================================
+--- ots-common.gypi (revision 83)
++++ ots-common.gypi (working copy)
+@@ -68,6 +68,7 @@
+ ],
+ 'ots_include_dirs': [
+ 'include',
++ '.',
+ ],
+ },
+ }
+Index: ots-standalone.gyp
+===================================================================
+--- ots-standalone.gyp (revision 83)
++++ ots-standalone.gyp (working copy)
+@@ -96,6 +96,12 @@
+ '<@(ots_include_dirs)',
+ ],
+ },
++ 'dependencies': [
++ 'third_party/lzma_sdk/lzma_sdk.gyp:lzma_sdk',
++ ],
++ 'export_dependent_settings': [
++ 'third_party/lzma_sdk/lzma_sdk.gyp:lzma_sdk',
++ ],
+ },
+ {
+ 'target_name': 'idempotent',
+@@ -117,5 +123,15 @@
+ }],
+ ],
+ },
++ {
++ 'target_name': 'lzma',
++ 'type': 'executable',
++ 'sources': [
++ 'test/lzma.cc',
++ ],
++ 'dependencies': [
++ 'ots',
++ ],
++ },
+ ],
+ }
+Index: third_party/lzma_sdk/LzmaEnc.h
+===================================================================
+--- third_party/lzma_sdk/LzmaEnc.h (revision 0)
++++ third_party/lzma_sdk/LzmaEnc.h (revision 0)
+@@ -0,0 +1,80 @@
++/* LzmaEnc.h -- LZMA Encoder
++2009-02-07 : Igor Pavlov : Public domain */
++
++#ifndef __LZMA_ENC_H
++#define __LZMA_ENC_H
++
++#include "Types.h"
++
++#ifdef __cplusplus
++extern "C" {
++#endif
++
++#define LZMA_PROPS_SIZE 5
++
++typedef struct _CLzmaEncProps
++{
++ int level; /* 0 <= level <= 9 */
++ UInt32 dictSize; /* (1 << 12) <= dictSize <= (1 << 27) for 32-bit version
++ (1 << 12) <= dictSize <= (1 << 30) for 64-bit version
++ default = (1 << 24) */
++ int lc; /* 0 <= lc <= 8, default = 3 */
++ int lp; /* 0 <= lp <= 4, default = 0 */
++ int pb; /* 0 <= pb <= 4, default = 2 */
++ int algo; /* 0 - fast, 1 - normal, default = 1 */
++ int fb; /* 5 <= fb <= 273, default = 32 */
++ int btMode; /* 0 - hashChain Mode, 1 - binTree mode - normal, default = 1 */
++ int numHashBytes; /* 2, 3 or 4, default = 4 */
++ UInt32 mc; /* 1 <= mc <= (1 << 30), default = 32 */
++ unsigned writeEndMark; /* 0 - do not write EOPM, 1 - write EOPM, default = 0 */
++ int numThreads; /* 1 or 2, default = 2 */
++} CLzmaEncProps;
++
++void LzmaEncProps_Init(CLzmaEncProps *p);
++void LzmaEncProps_Normalize(CLzmaEncProps *p);
++UInt32 LzmaEncProps_GetDictSize(const CLzmaEncProps *props2);
++
++
++/* ---------- CLzmaEncHandle Interface ---------- */
++
++/* LzmaEnc_* functions can return the following exit codes:
++Returns:
++ SZ_OK - OK
++ SZ_ERROR_MEM - Memory allocation error
++ SZ_ERROR_PARAM - Incorrect paramater in props
++ SZ_ERROR_WRITE - Write callback error.
++ SZ_ERROR_PROGRESS - some break from progress callback
++ SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version)
++*/
++
++typedef void * CLzmaEncHandle;
++
++CLzmaEncHandle LzmaEnc_Create(ISzAlloc *alloc);
++void LzmaEnc_Destroy(CLzmaEncHandle p, ISzAlloc *alloc, ISzAlloc *allocBig);
++SRes LzmaEnc_SetProps(CLzmaEncHandle p, const CLzmaEncProps *props);
++SRes LzmaEnc_WriteProperties(CLzmaEncHandle p, Byte *properties, SizeT *size);
++SRes LzmaEnc_Encode(CLzmaEncHandle p, ISeqOutStream *outStream, ISeqInStream *inStream,
++ ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig);
++SRes LzmaEnc_MemEncode(CLzmaEncHandle p, Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
++ int writeEndMark, ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig);
++
++/* ---------- One Call Interface ---------- */
++
++/* LzmaEncode
++Return code:
++ SZ_OK - OK
++ SZ_ERROR_MEM - Memory allocation error
++ SZ_ERROR_PARAM - Incorrect paramater
++ SZ_ERROR_OUTPUT_EOF - output buffer overflow
++ SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version)
++*/
++
++SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
++ const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeEndMark,
++ ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig);
++
++#ifdef __cplusplus
++}
++#endif
++
++#endif
+
+Property changes on: third_party/lzma_sdk/LzmaEnc.h
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzHash.h
+===================================================================
+--- third_party/lzma_sdk/LzHash.h (revision 0)
++++ third_party/lzma_sdk/LzHash.h (revision 0)
+@@ -0,0 +1,54 @@
++/* LzHash.h -- HASH functions for LZ algorithms
++2009-02-07 : Igor Pavlov : Public domain */
++
++#ifndef __LZ_HASH_H
++#define __LZ_HASH_H
++
++#define kHash2Size (1 << 10)
++#define kHash3Size (1 << 16)
++#define kHash4Size (1 << 20)
++
++#define kFix3HashSize (kHash2Size)
++#define kFix4HashSize (kHash2Size + kHash3Size)
++#define kFix5HashSize (kHash2Size + kHash3Size + kHash4Size)
++
++#define HASH2_CALC hashValue = cur[0] | ((UInt32)cur[1] << 8);
++
++#define HASH3_CALC { \
++ UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
++ hash2Value = temp & (kHash2Size - 1); \
++ hashValue = (temp ^ ((UInt32)cur[2] << 8)) & p->hashMask; }
++
++#define HASH4_CALC { \
++ UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
++ hash2Value = temp & (kHash2Size - 1); \
++ hash3Value = (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); \
++ hashValue = (temp ^ ((UInt32)cur[2] << 8) ^ (p->crc[cur[3]] << 5)) & p->hashMask; }
++
++#define HASH5_CALC { \
++ UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
++ hash2Value = temp & (kHash2Size - 1); \
++ hash3Value = (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); \
++ hash4Value = (temp ^ ((UInt32)cur[2] << 8) ^ (p->crc[cur[3]] << 5)); \
++ hashValue = (hash4Value ^ (p->crc[cur[4]] << 3)) & p->hashMask; \
++ hash4Value &= (kHash4Size - 1); }
++
++/* #define HASH_ZIP_CALC hashValue = ((cur[0] | ((UInt32)cur[1] << 8)) ^ p->crc[cur[2]]) & 0xFFFF; */
++#define HASH_ZIP_CALC hashValue = ((cur[2] | ((UInt32)cur[0] << 8)) ^ p->crc[cur[1]]) & 0xFFFF;
++
++
++#define MT_HASH2_CALC \
++ hash2Value = (p->crc[cur[0]] ^ cur[1]) & (kHash2Size - 1);
++
++#define MT_HASH3_CALC { \
++ UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
++ hash2Value = temp & (kHash2Size - 1); \
++ hash3Value = (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); }
++
++#define MT_HASH4_CALC { \
++ UInt32 temp = p->crc[cur[0]] ^ cur[1]; \
++ hash2Value = temp & (kHash2Size - 1); \
++ hash3Value = (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); \
++ hash4Value = (temp ^ ((UInt32)cur[2] << 8) ^ (p->crc[cur[3]] << 5)) & (kHash4Size - 1); }
++
++#endif
+
+Property changes on: third_party/lzma_sdk/LzHash.h
+___________________________________________________________________
+Added: svn:executable
+ + *
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/Alloc.h
+===================================================================
+--- third_party/lzma_sdk/Alloc.h (revision 0)
++++ third_party/lzma_sdk/Alloc.h (revision 0)
+@@ -0,0 +1,38 @@
++/* Alloc.h -- Memory allocation functions
++2009-02-07 : Igor Pavlov : Public domain */
++
++#ifndef __COMMON_ALLOC_H
++#define __COMMON_ALLOC_H
++
++#include <stddef.h>
++
++#ifdef __cplusplus
++extern "C" {
++#endif
++
++void *MyAlloc(size_t size);
++void MyFree(void *address);
++
++#ifdef _WIN32
++
++void SetLargePageSize();
++
++void *MidAlloc(size_t size);
++void MidFree(void *address);
++void *BigAlloc(size_t size);
++void BigFree(void *address);
++
++#else
++
++#define MidAlloc(size) MyAlloc(size)
++#define MidFree(address) MyFree(address)
++#define BigAlloc(size) MyAlloc(size)
++#define BigFree(address) MyFree(address)
++
++#endif
++
++#ifdef __cplusplus
++}
++#endif
++
++#endif
+
+Property changes on: third_party/lzma_sdk/Alloc.h
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzmaLib.h
+===================================================================
+--- third_party/lzma_sdk/LzmaLib.h (revision 0)
++++ third_party/lzma_sdk/LzmaLib.h (revision 0)
+@@ -0,0 +1,135 @@
++/* LzmaLib.h -- LZMA library interface
++2009-04-07 : Igor Pavlov : Public domain */
++
++#ifndef __LZMA_LIB_H
++#define __LZMA_LIB_H
++
++#include "Types.h"
++
++#ifdef __cplusplus
++extern "C" {
++#endif
++
++#define MY_STDAPI int MY_STD_CALL
++
++#define LZMA_PROPS_SIZE 5
++
++/*
++RAM requirements for LZMA:
++ for compression: (dictSize * 11.5 + 6 MB) + state_size
++ for decompression: dictSize + state_size
++ state_size = (4 + (1.5 << (lc + lp))) KB
++ by default (lc=3, lp=0), state_size = 16 KB.
++
++LZMA properties (5 bytes) format
++ Offset Size Description
++ 0 1 lc, lp and pb in encoded form.
++ 1 4 dictSize (little endian).
++*/
++
++/*
++LzmaCompress
++------------
++
++outPropsSize -
++ In: the pointer to the size of outProps buffer; *outPropsSize = LZMA_PROPS_SIZE = 5.
++ Out: the pointer to the size of written properties in outProps buffer; *outPropsSize = LZMA_PROPS_SIZE = 5.
++
++ LZMA Encoder will use defult values for any parameter, if it is
++ -1 for any from: level, loc, lp, pb, fb, numThreads
++ 0 for dictSize
++
++level - compression level: 0 <= level <= 9;
++
++ level dictSize algo fb
++ 0: 16 KB 0 32
++ 1: 64 KB 0 32
++ 2: 256 KB 0 32
++ 3: 1 MB 0 32
++ 4: 4 MB 0 32
++ 5: 16 MB 1 32
++ 6: 32 MB 1 32
++ 7+: 64 MB 1 64
++
++ The default value for "level" is 5.
++
++ algo = 0 means fast method
++ algo = 1 means normal method
++
++dictSize - The dictionary size in bytes. The maximum value is
++ 128 MB = (1 << 27) bytes for 32-bit version
++ 1 GB = (1 << 30) bytes for 64-bit version
++ The default value is 16 MB = (1 << 24) bytes.
++ It's recommended to use the dictionary that is larger than 4 KB and
++ that can be calculated as (1 << N) or (3 << N) sizes.
++
++lc - The number of literal context bits (high bits of previous literal).
++ It can be in the range from 0 to 8. The default value is 3.
++ Sometimes lc=4 gives the gain for big files.
++
++lp - The number of literal pos bits (low bits of current position for literals).
++ It can be in the range from 0 to 4. The default value is 0.
++ The lp switch is intended for periodical data when the period is equal to 2^lp.
++ For example, for 32-bit (4 bytes) periodical data you can use lp=2. Often it's
++ better to set lc=0, if you change lp switch.
++
++pb - The number of pos bits (low bits of current position).
++ It can be in the range from 0 to 4. The default value is 2.
++ The pb switch is intended for periodical data when the period is equal 2^pb.
++
++fb - Word size (the number of fast bytes).
++ It can be in the range from 5 to 273. The default value is 32.
++ Usually, a big number gives a little bit better compression ratio and
++ slower compression process.
++
++numThreads - The number of thereads. 1 or 2. The default value is 2.
++ Fast mode (algo = 0) can use only 1 thread.
++
++Out:
++ destLen - processed output size
++Returns:
++ SZ_OK - OK
++ SZ_ERROR_MEM - Memory allocation error
++ SZ_ERROR_PARAM - Incorrect paramater
++ SZ_ERROR_OUTPUT_EOF - output buffer overflow
++ SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version)
++*/
++
++MY_STDAPI LzmaCompress(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t srcLen,
++ unsigned char *outProps, size_t *outPropsSize, /* *outPropsSize must be = 5 */
++ int level, /* 0 <= level <= 9, default = 5 */
++ unsigned dictSize, /* default = (1 << 24) */
++ int lc, /* 0 <= lc <= 8, default = 3 */
++ int lp, /* 0 <= lp <= 4, default = 0 */
++ int pb, /* 0 <= pb <= 4, default = 2 */
++ int fb, /* 5 <= fb <= 273, default = 32 */
++ int numThreads /* 1 or 2, default = 2 */
++ );
++
++/*
++LzmaUncompress
++--------------
++In:
++ dest - output data
++ destLen - output data size
++ src - input data
++ srcLen - input data size
++Out:
++ destLen - processed output size
++ srcLen - processed input size
++Returns:
++ SZ_OK - OK
++ SZ_ERROR_DATA - Data error
++ SZ_ERROR_MEM - Memory allocation arror
++ SZ_ERROR_UNSUPPORTED - Unsupported properties
++ SZ_ERROR_INPUT_EOF - it needs more bytes in input buffer (src)
++*/
++
++MY_STDAPI LzmaUncompress(unsigned char *dest, size_t *destLen, const unsigned char *src, SizeT *srcLen,
++ const unsigned char *props, size_t propsSize);
++
++#ifdef __cplusplus
++}
++#endif
++
++#endif
+
+Property changes on: third_party/lzma_sdk/LzmaLib.h
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LICENSE
+===================================================================
+--- third_party/lzma_sdk/LICENSE (revision 0)
++++ third_party/lzma_sdk/LICENSE (revision 0)
+@@ -0,0 +1 @@
++LZMA SDK is placed in the public domain.
+Index: third_party/lzma_sdk/Types.h
+===================================================================
+--- third_party/lzma_sdk/Types.h (revision 0)
++++ third_party/lzma_sdk/Types.h (revision 0)
+@@ -0,0 +1,254 @@
++/* Types.h -- Basic types
++2010-10-09 : Igor Pavlov : Public domain */
++
++#ifndef __7Z_TYPES_H
++#define __7Z_TYPES_H
++
++#include <stddef.h>
++
++#ifdef _WIN32
++#include <windows.h>
++#endif
++
++#ifndef EXTERN_C_BEGIN
++#ifdef __cplusplus
++#define EXTERN_C_BEGIN extern "C" {
++#define EXTERN_C_END }
++#else
++#define EXTERN_C_BEGIN
++#define EXTERN_C_END
++#endif
++#endif
++
++EXTERN_C_BEGIN
++
++#define SZ_OK 0
++
++#define SZ_ERROR_DATA 1
++#define SZ_ERROR_MEM 2
++#define SZ_ERROR_CRC 3
++#define SZ_ERROR_UNSUPPORTED 4
++#define SZ_ERROR_PARAM 5
++#define SZ_ERROR_INPUT_EOF 6
++#define SZ_ERROR_OUTPUT_EOF 7
++#define SZ_ERROR_READ 8
++#define SZ_ERROR_WRITE 9
++#define SZ_ERROR_PROGRESS 10
++#define SZ_ERROR_FAIL 11
++#define SZ_ERROR_THREAD 12
++
++#define SZ_ERROR_ARCHIVE 16
++#define SZ_ERROR_NO_ARCHIVE 17
++
++typedef int SRes;
++
++#ifdef _WIN32
++typedef DWORD WRes;
++#else
++typedef int WRes;
++#endif
++
++#ifndef RINOK
++#define RINOK(x) { int __result__ = (x); if (__result__ != 0) return __result__; }
++#endif
++
++typedef unsigned char Byte;
++typedef short Int16;
++typedef unsigned short UInt16;
++
++#ifdef _LZMA_UINT32_IS_ULONG
++typedef long Int32;
++typedef unsigned long UInt32;
++#else
++typedef int Int32;
++typedef unsigned int UInt32;
++#endif
++
++#ifdef _SZ_NO_INT_64
++
++/* define _SZ_NO_INT_64, if your compiler doesn't support 64-bit integers.
++ NOTES: Some code will work incorrectly in that case! */
++
++typedef long Int64;
++typedef unsigned long UInt64;
++
++#else
++
++#if defined(_MSC_VER) || defined(__BORLANDC__)
++typedef __int64 Int64;
++typedef unsigned __int64 UInt64;
++#define UINT64_CONST(n) n
++#else
++typedef long long int Int64;
++typedef unsigned long long int UInt64;
++#define UINT64_CONST(n) n ## ULL
++#endif
++
++#endif
++
++#ifdef _LZMA_NO_SYSTEM_SIZE_T
++typedef UInt32 SizeT;
++#else
++typedef size_t SizeT;
++#endif
++
++typedef int Bool;
++#define True 1
++#define False 0
++
++
++#ifdef _WIN32
++#define MY_STD_CALL __stdcall
++#else
++#define MY_STD_CALL
++#endif
++
++#ifdef _MSC_VER
++
++#if _MSC_VER >= 1300
++#define MY_NO_INLINE __declspec(noinline)
++#else
++#define MY_NO_INLINE
++#endif
++
++#define MY_CDECL __cdecl
++#define MY_FAST_CALL __fastcall
++
++#else
++
++#define MY_CDECL
++#define MY_FAST_CALL
++
++#endif
++
++
++/* The following interfaces use first parameter as pointer to structure */
++
++typedef struct
++{
++ Byte (*Read)(void *p); /* reads one byte, returns 0 in case of EOF or error */
++} IByteIn;
++
++typedef struct
++{
++ void (*Write)(void *p, Byte b);
++} IByteOut;
++
++typedef struct
++{
++ SRes (*Read)(void *p, void *buf, size_t *size);
++ /* if (input(*size) != 0 && output(*size) == 0) means end_of_stream.
++ (output(*size) < input(*size)) is allowed */
++} ISeqInStream;
++
++/* it can return SZ_ERROR_INPUT_EOF */
++SRes SeqInStream_Read(ISeqInStream *stream, void *buf, size_t size);
++SRes SeqInStream_Read2(ISeqInStream *stream, void *buf, size_t size, SRes errorType);
++SRes SeqInStream_ReadByte(ISeqInStream *stream, Byte *buf);
++
++typedef struct
++{
++ size_t (*Write)(void *p, const void *buf, size_t size);
++ /* Returns: result - the number of actually written bytes.
++ (result < size) means error */
++} ISeqOutStream;
++
++typedef enum
++{
++ SZ_SEEK_SET = 0,
++ SZ_SEEK_CUR = 1,
++ SZ_SEEK_END = 2
++} ESzSeek;
++
++typedef struct
++{
++ SRes (*Read)(void *p, void *buf, size_t *size); /* same as ISeqInStream::Read */
++ SRes (*Seek)(void *p, Int64 *pos, ESzSeek origin);
++} ISeekInStream;
++
++typedef struct
++{
++ SRes (*Look)(void *p, const void **buf, size_t *size);
++ /* if (input(*size) != 0 && output(*size) == 0) means end_of_stream.
++ (output(*size) > input(*size)) is not allowed
++ (output(*size) < input(*size)) is allowed */
++ SRes (*Skip)(void *p, size_t offset);
++ /* offset must be <= output(*size) of Look */
++
++ SRes (*Read)(void *p, void *buf, size_t *size);
++ /* reads directly (without buffer). It's same as ISeqInStream::Read */
++ SRes (*Seek)(void *p, Int64 *pos, ESzSeek origin);
++} ILookInStream;
++
++SRes LookInStream_LookRead(ILookInStream *stream, void *buf, size_t *size);
++SRes LookInStream_SeekTo(ILookInStream *stream, UInt64 offset);
++
++/* reads via ILookInStream::Read */
++SRes LookInStream_Read2(ILookInStream *stream, void *buf, size_t size, SRes errorType);
++SRes LookInStream_Read(ILookInStream *stream, void *buf, size_t size);
++
++#define LookToRead_BUF_SIZE (1 << 14)
++
++typedef struct
++{
++ ILookInStream s;
++ ISeekInStream *realStream;
++ size_t pos;
++ size_t size;
++ Byte buf[LookToRead_BUF_SIZE];
++} CLookToRead;
++
++void LookToRead_CreateVTable(CLookToRead *p, int lookahead);
++void LookToRead_Init(CLookToRead *p);
++
++typedef struct
++{
++ ISeqInStream s;
++ ILookInStream *realStream;
++} CSecToLook;
++
++void SecToLook_CreateVTable(CSecToLook *p);
++
++typedef struct
++{
++ ISeqInStream s;
++ ILookInStream *realStream;
++} CSecToRead;
++
++void SecToRead_CreateVTable(CSecToRead *p);
++
++typedef struct
++{
++ SRes (*Progress)(void *p, UInt64 inSize, UInt64 outSize);
++ /* Returns: result. (result != SZ_OK) means break.
++ Value (UInt64)(Int64)-1 for size means unknown value. */
++} ICompressProgress;
++
++typedef struct
++{
++ void *(*Alloc)(void *p, size_t size);
++ void (*Free)(void *p, void *address); /* address can be 0 */
++} ISzAlloc;
++
++#define IAlloc_Alloc(p, size) (p)->Alloc((p), size)
++#define IAlloc_Free(p, a) (p)->Free((p), a)
++
++#ifdef _WIN32
++
++#define CHAR_PATH_SEPARATOR '\\'
++#define WCHAR_PATH_SEPARATOR L'\\'
++#define STRING_PATH_SEPARATOR "\\"
++#define WSTRING_PATH_SEPARATOR L"\\"
++
++#else
++
++#define CHAR_PATH_SEPARATOR '/'
++#define WCHAR_PATH_SEPARATOR L'/'
++#define STRING_PATH_SEPARATOR "/"
++#define WSTRING_PATH_SEPARATOR L"/"
++
++#endif
++
++EXTERN_C_END
++
++#endif
+
+Property changes on: third_party/lzma_sdk/Types.h
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzmaDec.c
+===================================================================
+--- third_party/lzma_sdk/LzmaDec.c (revision 0)
++++ third_party/lzma_sdk/LzmaDec.c (revision 0)
+@@ -0,0 +1,999 @@
++/* LzmaDec.c -- LZMA Decoder
++2009-09-20 : Igor Pavlov : Public domain */
++
++#include "LzmaDec.h"
++
++#include <string.h>
++
++#define kNumTopBits 24
++#define kTopValue ((UInt32)1 << kNumTopBits)
++
++#define kNumBitModelTotalBits 11
++#define kBitModelTotal (1 << kNumBitModelTotalBits)
++#define kNumMoveBits 5
++
++#define RC_INIT_SIZE 5
++
++#define NORMALIZE if (range < kTopValue) { range <<= 8; code = (code << 8) | (*buf++); }
++
++#define IF_BIT_0(p) ttt = *(p); NORMALIZE; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
++#define UPDATE_0(p) range = bound; *(p) = (CLzmaProb)(ttt + ((kBitModelTotal - ttt) >> kNumMoveBits));
++#define UPDATE_1(p) range -= bound; code -= bound; *(p) = (CLzmaProb)(ttt - (ttt >> kNumMoveBits));
++#define GET_BIT2(p, i, A0, A1) IF_BIT_0(p) \
++ { UPDATE_0(p); i = (i + i); A0; } else \
++ { UPDATE_1(p); i = (i + i) + 1; A1; }
++#define GET_BIT(p, i) GET_BIT2(p, i, ; , ;)
++
++#define TREE_GET_BIT(probs, i) { GET_BIT((probs + i), i); }
++#define TREE_DECODE(probs, limit, i) \
++ { i = 1; do { TREE_GET_BIT(probs, i); } while (i < limit); i -= limit; }
++
++/* #define _LZMA_SIZE_OPT */
++
++#ifdef _LZMA_SIZE_OPT
++#define TREE_6_DECODE(probs, i) TREE_DECODE(probs, (1 << 6), i)
++#else
++#define TREE_6_DECODE(probs, i) \
++ { i = 1; \
++ TREE_GET_BIT(probs, i); \
++ TREE_GET_BIT(probs, i); \
++ TREE_GET_BIT(probs, i); \
++ TREE_GET_BIT(probs, i); \
++ TREE_GET_BIT(probs, i); \
++ TREE_GET_BIT(probs, i); \
++ i -= 0x40; }
++#endif
++
++#define NORMALIZE_CHECK if (range < kTopValue) { if (buf >= bufLimit) return DUMMY_ERROR; range <<= 8; code = (code << 8) | (*buf++); }
++
++#define IF_BIT_0_CHECK(p) ttt = *(p); NORMALIZE_CHECK; bound = (range >> kNumBitModelTotalBits) * ttt; if (code < bound)
++#define UPDATE_0_CHECK range = bound;
++#define UPDATE_1_CHECK range -= bound; code -= bound;
++#define GET_BIT2_CHECK(p, i, A0, A1) IF_BIT_0_CHECK(p) \
++ { UPDATE_0_CHECK; i = (i + i); A0; } else \
++ { UPDATE_1_CHECK; i = (i + i) + 1; A1; }
++#define GET_BIT_CHECK(p, i) GET_BIT2_CHECK(p, i, ; , ;)
++#define TREE_DECODE_CHECK(probs, limit, i) \
++ { i = 1; do { GET_BIT_CHECK(probs + i, i) } while (i < limit); i -= limit; }
++
++
++#define kNumPosBitsMax 4
++#define kNumPosStatesMax (1 << kNumPosBitsMax)
++
++#define kLenNumLowBits 3
++#define kLenNumLowSymbols (1 << kLenNumLowBits)
++#define kLenNumMidBits 3
++#define kLenNumMidSymbols (1 << kLenNumMidBits)
++#define kLenNumHighBits 8
++#define kLenNumHighSymbols (1 << kLenNumHighBits)
++
++#define LenChoice 0
++#define LenChoice2 (LenChoice + 1)
++#define LenLow (LenChoice2 + 1)
++#define LenMid (LenLow + (kNumPosStatesMax << kLenNumLowBits))
++#define LenHigh (LenMid + (kNumPosStatesMax << kLenNumMidBits))
++#define kNumLenProbs (LenHigh + kLenNumHighSymbols)
++
++
++#define kNumStates 12
++#define kNumLitStates 7
++
++#define kStartPosModelIndex 4
++#define kEndPosModelIndex 14
++#define kNumFullDistances (1 << (kEndPosModelIndex >> 1))
++
++#define kNumPosSlotBits 6
++#define kNumLenToPosStates 4
++
++#define kNumAlignBits 4
++#define kAlignTableSize (1 << kNumAlignBits)
++
++#define kMatchMinLen 2
++#define kMatchSpecLenStart (kMatchMinLen + kLenNumLowSymbols + kLenNumMidSymbols + kLenNumHighSymbols)
++
++#define IsMatch 0
++#define IsRep (IsMatch + (kNumStates << kNumPosBitsMax))
++#define IsRepG0 (IsRep + kNumStates)
++#define IsRepG1 (IsRepG0 + kNumStates)
++#define IsRepG2 (IsRepG1 + kNumStates)
++#define IsRep0Long (IsRepG2 + kNumStates)
++#define PosSlot (IsRep0Long + (kNumStates << kNumPosBitsMax))
++#define SpecPos (PosSlot + (kNumLenToPosStates << kNumPosSlotBits))
++#define Align (SpecPos + kNumFullDistances - kEndPosModelIndex)
++#define LenCoder (Align + kAlignTableSize)
++#define RepLenCoder (LenCoder + kNumLenProbs)
++#define Literal (RepLenCoder + kNumLenProbs)
++
++#define LZMA_BASE_SIZE 1846
++#define LZMA_LIT_SIZE 768
++
++#define LzmaProps_GetNumProbs(p) ((UInt32)LZMA_BASE_SIZE + (LZMA_LIT_SIZE << ((p)->lc + (p)->lp)))
++
++#if Literal != LZMA_BASE_SIZE
++StopCompilingDueBUG
++#endif
++
++#define LZMA_DIC_MIN (1 << 12)
++
++/* First LZMA-symbol is always decoded.
++And it decodes new LZMA-symbols while (buf < bufLimit), but "buf" is without last normalization
++Out:
++ Result:
++ SZ_OK - OK
++ SZ_ERROR_DATA - Error
++ p->remainLen:
++ < kMatchSpecLenStart : normal remain
++ = kMatchSpecLenStart : finished
++ = kMatchSpecLenStart + 1 : Flush marker
++ = kMatchSpecLenStart + 2 : State Init Marker
++*/
++
++static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *p, SizeT limit, const Byte *bufLimit)
++{
++ CLzmaProb *probs = p->probs;
++
++ unsigned state = p->state;
++ UInt32 rep0 = p->reps[0], rep1 = p->reps[1], rep2 = p->reps[2], rep3 = p->reps[3];
++ unsigned pbMask = ((unsigned)1 << (p->prop.pb)) - 1;
++ unsigned lpMask = ((unsigned)1 << (p->prop.lp)) - 1;
++ unsigned lc = p->prop.lc;
++
++ Byte *dic = p->dic;
++ SizeT dicBufSize = p->dicBufSize;
++ SizeT dicPos = p->dicPos;
++
++ UInt32 processedPos = p->processedPos;
++ UInt32 checkDicSize = p->checkDicSize;
++ unsigned len = 0;
++
++ const Byte *buf = p->buf;
++ UInt32 range = p->range;
++ UInt32 code = p->code;
++
++ do
++ {
++ CLzmaProb *prob;
++ UInt32 bound;
++ unsigned ttt;
++ unsigned posState = processedPos & pbMask;
++
++ prob = probs + IsMatch + (state << kNumPosBitsMax) + posState;
++ IF_BIT_0(prob)
++ {
++ unsigned symbol;
++ UPDATE_0(prob);
++ prob = probs + Literal;
++ if (checkDicSize != 0 || processedPos != 0)
++ prob += (LZMA_LIT_SIZE * (((processedPos & lpMask) << lc) +
++ (dic[(dicPos == 0 ? dicBufSize : dicPos) - 1] >> (8 - lc))));
++
++ if (state < kNumLitStates)
++ {
++ state -= (state < 4) ? state : 3;
++ symbol = 1;
++ do { GET_BIT(prob + symbol, symbol) } while (symbol < 0x100);
++ }
++ else
++ {
++ unsigned matchByte = p->dic[(dicPos - rep0) + ((dicPos < rep0) ? dicBufSize : 0)];
++ unsigned offs = 0x100;
++ state -= (state < 10) ? 3 : 6;
++ symbol = 1;
++ do
++ {
++ unsigned bit;
++ CLzmaProb *probLit;
++ matchByte <<= 1;
++ bit = (matchByte & offs);
++ probLit = prob + offs + bit + symbol;
++ GET_BIT2(probLit, symbol, offs &= ~bit, offs &= bit)
++ }
++ while (symbol < 0x100);
++ }
++ dic[dicPos++] = (Byte)symbol;
++ processedPos++;
++ continue;
++ }
++ else
++ {
++ UPDATE_1(prob);
++ prob = probs + IsRep + state;
++ IF_BIT_0(prob)
++ {
++ UPDATE_0(prob);
++ state += kNumStates;
++ prob = probs + LenCoder;
++ }
++ else
++ {
++ UPDATE_1(prob);
++ if (checkDicSize == 0 && processedPos == 0)
++ return SZ_ERROR_DATA;
++ prob = probs + IsRepG0 + state;
++ IF_BIT_0(prob)
++ {
++ UPDATE_0(prob);
++ prob = probs + IsRep0Long + (state << kNumPosBitsMax) + posState;
++ IF_BIT_0(prob)
++ {
++ UPDATE_0(prob);
++ dic[dicPos] = dic[(dicPos - rep0) + ((dicPos < rep0) ? dicBufSize : 0)];
++ dicPos++;
++ processedPos++;
++ state = state < kNumLitStates ? 9 : 11;
++ continue;
++ }
++ UPDATE_1(prob);
++ }
++ else
++ {
++ UInt32 distance;
++ UPDATE_1(prob);
++ prob = probs + IsRepG1 + state;
++ IF_BIT_0(prob)
++ {
++ UPDATE_0(prob);
++ distance = rep1;
++ }
++ else
++ {
++ UPDATE_1(prob);
++ prob = probs + IsRepG2 + state;
++ IF_BIT_0(prob)
++ {
++ UPDATE_0(prob);
++ distance = rep2;
++ }
++ else
++ {
++ UPDATE_1(prob);
++ distance = rep3;
++ rep3 = rep2;
++ }
++ rep2 = rep1;
++ }
++ rep1 = rep0;
++ rep0 = distance;
++ }
++ state = state < kNumLitStates ? 8 : 11;
++ prob = probs + RepLenCoder;
++ }
++ {
++ unsigned limit, offset;
++ CLzmaProb *probLen = prob + LenChoice;
++ IF_BIT_0(probLen)
++ {
++ UPDATE_0(probLen);
++ probLen = prob + LenLow + (posState << kLenNumLowBits);
++ offset = 0;
++ limit = (1 << kLenNumLowBits);
++ }
++ else
++ {
++ UPDATE_1(probLen);
++ probLen = prob + LenChoice2;
++ IF_BIT_0(probLen)
++ {
++ UPDATE_0(probLen);
++ probLen = prob + LenMid + (posState << kLenNumMidBits);
++ offset = kLenNumLowSymbols;
++ limit = (1 << kLenNumMidBits);
++ }
++ else
++ {
++ UPDATE_1(probLen);
++ probLen = prob + LenHigh;
++ offset = kLenNumLowSymbols + kLenNumMidSymbols;
++ limit = (1 << kLenNumHighBits);
++ }
++ }
++ TREE_DECODE(probLen, limit, len);
++ len += offset;
++ }
++
++ if (state >= kNumStates)
++ {
++ UInt32 distance;
++ prob = probs + PosSlot +
++ ((len < kNumLenToPosStates ? len : kNumLenToPosStates - 1) << kNumPosSlotBits);
++ TREE_6_DECODE(prob, distance);
++ if (distance >= kStartPosModelIndex)
++ {
++ unsigned posSlot = (unsigned)distance;
++ int numDirectBits = (int)(((distance >> 1) - 1));
++ distance = (2 | (distance & 1));
++ if (posSlot < kEndPosModelIndex)
++ {
++ distance <<= numDirectBits;
++ prob = probs + SpecPos + distance - posSlot - 1;
++ {
++ UInt32 mask = 1;
++ unsigned i = 1;
++ do
++ {
++ GET_BIT2(prob + i, i, ; , distance |= mask);
++ mask <<= 1;
++ }
++ while (--numDirectBits != 0);
++ }
++ }
++ else
++ {
++ numDirectBits -= kNumAlignBits;
++ do
++ {
++ NORMALIZE
++ range >>= 1;
++
++ {
++ UInt32 t;
++ code -= range;
++ t = (0 - ((UInt32)code >> 31)); /* (UInt32)((Int32)code >> 31) */
++ distance = (distance << 1) + (t + 1);
++ code += range & t;
++ }
++ /*
++ distance <<= 1;
++ if (code >= range)
++ {
++ code -= range;
++ distance |= 1;
++ }
++ */
++ }
++ while (--numDirectBits != 0);
++ prob = probs + Align;
++ distance <<= kNumAlignBits;
++ {
++ unsigned i = 1;
++ GET_BIT2(prob + i, i, ; , distance |= 1);
++ GET_BIT2(prob + i, i, ; , distance |= 2);
++ GET_BIT2(prob + i, i, ; , distance |= 4);
++ GET_BIT2(prob + i, i, ; , distance |= 8);
++ }
++ if (distance == (UInt32)0xFFFFFFFF)
++ {
++ len += kMatchSpecLenStart;
++ state -= kNumStates;
++ break;
++ }
++ }
++ }
++ rep3 = rep2;
++ rep2 = rep1;
++ rep1 = rep0;
++ rep0 = distance + 1;
++ if (checkDicSize == 0)
++ {
++ if (distance >= processedPos)
++ return SZ_ERROR_DATA;
++ }
++ else if (distance >= checkDicSize)
++ return SZ_ERROR_DATA;
++ state = (state < kNumStates + kNumLitStates) ? kNumLitStates : kNumLitStates + 3;
++ }
++
++ len += kMatchMinLen;
++
++ if (limit == dicPos)
++ return SZ_ERROR_DATA;
++ {
++ SizeT rem = limit - dicPos;
++ unsigned curLen = ((rem < len) ? (unsigned)rem : len);
++ SizeT pos = (dicPos - rep0) + ((dicPos < rep0) ? dicBufSize : 0);
++
++ processedPos += curLen;
++
++ len -= curLen;
++ if (pos + curLen <= dicBufSize)
++ {
++ Byte *dest = dic + dicPos;
++ ptrdiff_t src = (ptrdiff_t)pos - (ptrdiff_t)dicPos;
++ const Byte *lim = dest + curLen;
++ dicPos += curLen;
++ do
++ *(dest) = (Byte)*(dest + src);
++ while (++dest != lim);
++ }
++ else
++ {
++ do
++ {
++ dic[dicPos++] = dic[pos];
++ if (++pos == dicBufSize)
++ pos = 0;
++ }
++ while (--curLen != 0);
++ }
++ }
++ }
++ }
++ while (dicPos < limit && buf < bufLimit);
++ NORMALIZE;
++ p->buf = buf;
++ p->range = range;
++ p->code = code;
++ p->remainLen = len;
++ p->dicPos = dicPos;
++ p->processedPos = processedPos;
++ p->reps[0] = rep0;
++ p->reps[1] = rep1;
++ p->reps[2] = rep2;
++ p->reps[3] = rep3;
++ p->state = state;
++
++ return SZ_OK;
++}
++
++static void MY_FAST_CALL LzmaDec_WriteRem(CLzmaDec *p, SizeT limit)
++{
++ if (p->remainLen != 0 && p->remainLen < kMatchSpecLenStart)
++ {
++ Byte *dic = p->dic;
++ SizeT dicPos = p->dicPos;
++ SizeT dicBufSize = p->dicBufSize;
++ unsigned len = p->remainLen;
++ UInt32 rep0 = p->reps[0];
++ if (limit - dicPos < len)
++ len = (unsigned)(limit - dicPos);
++
++ if (p->checkDicSize == 0 && p->prop.dicSize - p->processedPos <= len)
++ p->checkDicSize = p->prop.dicSize;
++
++ p->processedPos += len;
++ p->remainLen -= len;
++ while (len-- != 0)
++ {
++ dic[dicPos] = dic[(dicPos - rep0) + ((dicPos < rep0) ? dicBufSize : 0)];
++ dicPos++;
++ }
++ p->dicPos = dicPos;
++ }
++}
++
++static int MY_FAST_CALL LzmaDec_DecodeReal2(CLzmaDec *p, SizeT limit, const Byte *bufLimit)
++{
++ do
++ {
++ SizeT limit2 = limit;
++ if (p->checkDicSize == 0)
++ {
++ UInt32 rem = p->prop.dicSize - p->processedPos;
++ if (limit - p->dicPos > rem)
++ limit2 = p->dicPos + rem;
++ }
++ RINOK(LzmaDec_DecodeReal(p, limit2, bufLimit));
++ if (p->processedPos >= p->prop.dicSize)
++ p->checkDicSize = p->prop.dicSize;
++ LzmaDec_WriteRem(p, limit);
++ }
++ while (p->dicPos < limit && p->buf < bufLimit && p->remainLen < kMatchSpecLenStart);
++
++ if (p->remainLen > kMatchSpecLenStart)
++ {
++ p->remainLen = kMatchSpecLenStart;
++ }
++ return 0;
++}
++
++typedef enum
++{
++ DUMMY_ERROR, /* unexpected end of input stream */
++ DUMMY_LIT,
++ DUMMY_MATCH,
++ DUMMY_REP
++} ELzmaDummy;
++
++static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, const Byte *buf, SizeT inSize)
++{
++ UInt32 range = p->range;
++ UInt32 code = p->code;
++ const Byte *bufLimit = buf + inSize;
++ CLzmaProb *probs = p->probs;
++ unsigned state = p->state;
++ ELzmaDummy res;
++
++ {
++ CLzmaProb *prob;
++ UInt32 bound;
++ unsigned ttt;
++ unsigned posState = (p->processedPos) & ((1 << p->prop.pb) - 1);
++
++ prob = probs + IsMatch + (state << kNumPosBitsMax) + posState;
++ IF_BIT_0_CHECK(prob)
++ {
++ UPDATE_0_CHECK
++
++ /* if (bufLimit - buf >= 7) return DUMMY_LIT; */
++
++ prob = probs + Literal;
++ if (p->checkDicSize != 0 || p->processedPos != 0)
++ prob += (LZMA_LIT_SIZE *
++ ((((p->processedPos) & ((1 << (p->prop.lp)) - 1)) << p->prop.lc) +
++ (p->dic[(p->dicPos == 0 ? p->dicBufSize : p->dicPos) - 1] >> (8 - p->prop.lc))));
++
++ if (state < kNumLitStates)
++ {
++ unsigned symbol = 1;
++ do { GET_BIT_CHECK(prob + symbol, symbol) } while (symbol < 0x100);
++ }
++ else
++ {
++ unsigned matchByte = p->dic[p->dicPos - p->reps[0] +
++ ((p->dicPos < p->reps[0]) ? p->dicBufSize : 0)];
++ unsigned offs = 0x100;
++ unsigned symbol = 1;
++ do
++ {
++ unsigned bit;
++ CLzmaProb *probLit;
++ matchByte <<= 1;
++ bit = (matchByte & offs);
++ probLit = prob + offs + bit + symbol;
++ GET_BIT2_CHECK(probLit, symbol, offs &= ~bit, offs &= bit)
++ }
++ while (symbol < 0x100);
++ }
++ res = DUMMY_LIT;
++ }
++ else
++ {
++ unsigned len;
++ UPDATE_1_CHECK;
++
++ prob = probs + IsRep + state;
++ IF_BIT_0_CHECK(prob)
++ {
++ UPDATE_0_CHECK;
++ state = 0;
++ prob = probs + LenCoder;
++ res = DUMMY_MATCH;
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ res = DUMMY_REP;
++ prob = probs + IsRepG0 + state;
++ IF_BIT_0_CHECK(prob)
++ {
++ UPDATE_0_CHECK;
++ prob = probs + IsRep0Long + (state << kNumPosBitsMax) + posState;
++ IF_BIT_0_CHECK(prob)
++ {
++ UPDATE_0_CHECK;
++ NORMALIZE_CHECK;
++ return DUMMY_REP;
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ }
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ prob = probs + IsRepG1 + state;
++ IF_BIT_0_CHECK(prob)
++ {
++ UPDATE_0_CHECK;
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ prob = probs + IsRepG2 + state;
++ IF_BIT_0_CHECK(prob)
++ {
++ UPDATE_0_CHECK;
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ }
++ }
++ }
++ state = kNumStates;
++ prob = probs + RepLenCoder;
++ }
++ {
++ unsigned limit, offset;
++ CLzmaProb *probLen = prob + LenChoice;
++ IF_BIT_0_CHECK(probLen)
++ {
++ UPDATE_0_CHECK;
++ probLen = prob + LenLow + (posState << kLenNumLowBits);
++ offset = 0;
++ limit = 1 << kLenNumLowBits;
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ probLen = prob + LenChoice2;
++ IF_BIT_0_CHECK(probLen)
++ {
++ UPDATE_0_CHECK;
++ probLen = prob + LenMid + (posState << kLenNumMidBits);
++ offset = kLenNumLowSymbols;
++ limit = 1 << kLenNumMidBits;
++ }
++ else
++ {
++ UPDATE_1_CHECK;
++ probLen = prob + LenHigh;
++ offset = kLenNumLowSymbols + kLenNumMidSymbols;
++ limit = 1 << kLenNumHighBits;
++ }
++ }
++ TREE_DECODE_CHECK(probLen, limit, len);
++ len += offset;
++ }
++
++ if (state < 4)
++ {
++ unsigned posSlot;
++ prob = probs + PosSlot +
++ ((len < kNumLenToPosStates ? len : kNumLenToPosStates - 1) <<
++ kNumPosSlotBits);
++ TREE_DECODE_CHECK(prob, 1 << kNumPosSlotBits, posSlot);
++ if (posSlot >= kStartPosModelIndex)
++ {
++ int numDirectBits = ((posSlot >> 1) - 1);
++
++ /* if (bufLimit - buf >= 8) return DUMMY_MATCH; */
++
++ if (posSlot < kEndPosModelIndex)
++ {
++ prob = probs + SpecPos + ((2 | (posSlot & 1)) << numDirectBits) - posSlot - 1;
++ }
++ else
++ {
++ numDirectBits -= kNumAlignBits;
++ do
++ {
++ NORMALIZE_CHECK
++ range >>= 1;
++ code -= range & (((code - range) >> 31) - 1);
++ /* if (code >= range) code -= range; */
++ }
++ while (--numDirectBits != 0);
++ prob = probs + Align;
++ numDirectBits = kNumAlignBits;
++ }
++ {
++ unsigned i = 1;
++ do
++ {
++ GET_BIT_CHECK(prob + i, i);
++ }
++ while (--numDirectBits != 0);
++ }
++ }
++ }
++ }
++ }
++ NORMALIZE_CHECK;
++ return res;
++}
++
++
++static void LzmaDec_InitRc(CLzmaDec *p, const Byte *data)
++{
++ p->code = ((UInt32)data[1] << 24) | ((UInt32)data[2] << 16) | ((UInt32)data[3] << 8) | ((UInt32)data[4]);
++ p->range = 0xFFFFFFFF;
++ p->needFlush = 0;
++}
++
++void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState)
++{
++ p->needFlush = 1;
++ p->remainLen = 0;
++ p->tempBufSize = 0;
++
++ if (initDic)
++ {
++ p->processedPos = 0;
++ p->checkDicSize = 0;
++ p->needInitState = 1;
++ }
++ if (initState)
++ p->needInitState = 1;
++}
++
++void LzmaDec_Init(CLzmaDec *p)
++{
++ p->dicPos = 0;
++ LzmaDec_InitDicAndState(p, True, True);
++}
++
++static void LzmaDec_InitStateReal(CLzmaDec *p)
++{
++ UInt32 numProbs = Literal + ((UInt32)LZMA_LIT_SIZE << (p->prop.lc + p->prop.lp));
++ UInt32 i;
++ CLzmaProb *probs = p->probs;
++ for (i = 0; i < numProbs; i++)
++ probs[i] = kBitModelTotal >> 1;
++ p->reps[0] = p->reps[1] = p->reps[2] = p->reps[3] = 1;
++ p->state = 0;
++ p->needInitState = 0;
++}
++
++SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, const Byte *src, SizeT *srcLen,
++ ELzmaFinishMode finishMode, ELzmaStatus *status)
++{
++ SizeT inSize = *srcLen;
++ (*srcLen) = 0;
++ LzmaDec_WriteRem(p, dicLimit);
++
++ *status = LZMA_STATUS_NOT_SPECIFIED;
++
++ while (p->remainLen != kMatchSpecLenStart)
++ {
++ int checkEndMarkNow;
++
++ if (p->needFlush != 0)
++ {
++ for (; inSize > 0 && p->tempBufSize < RC_INIT_SIZE; (*srcLen)++, inSize--)
++ p->tempBuf[p->tempBufSize++] = *src++;
++ if (p->tempBufSize < RC_INIT_SIZE)
++ {
++ *status = LZMA_STATUS_NEEDS_MORE_INPUT;
++ return SZ_OK;
++ }
++ if (p->tempBuf[0] != 0)
++ return SZ_ERROR_DATA;
++
++ LzmaDec_InitRc(p, p->tempBuf);
++ p->tempBufSize = 0;
++ }
++
++ checkEndMarkNow = 0;
++ if (p->dicPos >= dicLimit)
++ {
++ if (p->remainLen == 0 && p->code == 0)
++ {
++ *status = LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK;
++ return SZ_OK;
++ }
++ if (finishMode == LZMA_FINISH_ANY)
++ {
++ *status = LZMA_STATUS_NOT_FINISHED;
++ return SZ_OK;
++ }
++ if (p->remainLen != 0)
++ {
++ *status = LZMA_STATUS_NOT_FINISHED;
++ return SZ_ERROR_DATA;
++ }
++ checkEndMarkNow = 1;
++ }
++
++ if (p->needInitState)
++ LzmaDec_InitStateReal(p);
++
++ if (p->tempBufSize == 0)
++ {
++ SizeT processed;
++ const Byte *bufLimit;
++ if (inSize < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
++ {
++ int dummyRes = LzmaDec_TryDummy(p, src, inSize);
++ if (dummyRes == DUMMY_ERROR)
++ {
++ memcpy(p->tempBuf, src, inSize);
++ p->tempBufSize = (unsigned)inSize;
++ (*srcLen) += inSize;
++ *status = LZMA_STATUS_NEEDS_MORE_INPUT;
++ return SZ_OK;
++ }
++ if (checkEndMarkNow && dummyRes != DUMMY_MATCH)
++ {
++ *status = LZMA_STATUS_NOT_FINISHED;
++ return SZ_ERROR_DATA;
++ }
++ bufLimit = src;
++ }
++ else
++ bufLimit = src + inSize - LZMA_REQUIRED_INPUT_MAX;
++ p->buf = src;
++ if (LzmaDec_DecodeReal2(p, dicLimit, bufLimit) != 0)
++ return SZ_ERROR_DATA;
++ processed = (SizeT)(p->buf - src);
++ (*srcLen) += processed;
++ src += processed;
++ inSize -= processed;
++ }
++ else
++ {
++ unsigned rem = p->tempBufSize, lookAhead = 0;
++ while (rem < LZMA_REQUIRED_INPUT_MAX && lookAhead < inSize)
++ p->tempBuf[rem++] = src[lookAhead++];
++ p->tempBufSize = rem;
++ if (rem < LZMA_REQUIRED_INPUT_MAX || checkEndMarkNow)
++ {
++ int dummyRes = LzmaDec_TryDummy(p, p->tempBuf, rem);
++ if (dummyRes == DUMMY_ERROR)
++ {
++ (*srcLen) += lookAhead;
++ *status = LZMA_STATUS_NEEDS_MORE_INPUT;
++ return SZ_OK;
++ }
++ if (checkEndMarkNow && dummyRes != DUMMY_MATCH)
++ {
++ *status = LZMA_STATUS_NOT_FINISHED;
++ return SZ_ERROR_DATA;
++ }
++ }
++ p->buf = p->tempBuf;
++ if (LzmaDec_DecodeReal2(p, dicLimit, p->buf) != 0)
++ return SZ_ERROR_DATA;
++ lookAhead -= (rem - (unsigned)(p->buf - p->tempBuf));
++ (*srcLen) += lookAhead;
++ src += lookAhead;
++ inSize -= lookAhead;
++ p->tempBufSize = 0;
++ }
++ }
++ if (p->code == 0)
++ *status = LZMA_STATUS_FINISHED_WITH_MARK;
++ return (p->code == 0) ? SZ_OK : SZ_ERROR_DATA;
++}
++
++SRes LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status)
++{
++ SizeT outSize = *destLen;
++ SizeT inSize = *srcLen;
++ *srcLen = *destLen = 0;
++ for (;;)
++ {
++ SizeT inSizeCur = inSize, outSizeCur, dicPos;
++ ELzmaFinishMode curFinishMode;
++ SRes res;
++ if (p->dicPos == p->dicBufSize)
++ p->dicPos = 0;
++ dicPos = p->dicPos;
++ if (outSize > p->dicBufSize - dicPos)
++ {
++ outSizeCur = p->dicBufSize;
++ curFinishMode = LZMA_FINISH_ANY;
++ }
++ else
++ {
++ outSizeCur = dicPos + outSize;
++ curFinishMode = finishMode;
++ }
++
++ res = LzmaDec_DecodeToDic(p, outSizeCur, src, &inSizeCur, curFinishMode, status);
++ src += inSizeCur;
++ inSize -= inSizeCur;
++ *srcLen += inSizeCur;
++ outSizeCur = p->dicPos - dicPos;
++ memcpy(dest, p->dic + dicPos, outSizeCur);
++ dest += outSizeCur;
++ outSize -= outSizeCur;
++ *destLen += outSizeCur;
++ if (res != 0)
++ return res;
++ if (outSizeCur == 0 || outSize == 0)
++ return SZ_OK;
++ }
++}
++
++void LzmaDec_FreeProbs(CLzmaDec *p, ISzAlloc *alloc)
++{
++ alloc->Free(alloc, p->probs);
++ p->probs = 0;
++}
++
++static void LzmaDec_FreeDict(CLzmaDec *p, ISzAlloc *alloc)
++{
++ alloc->Free(alloc, p->dic);
++ p->dic = 0;
++}
++
++void LzmaDec_Free(CLzmaDec *p, ISzAlloc *alloc)
++{
++ LzmaDec_FreeProbs(p, alloc);
++ LzmaDec_FreeDict(p, alloc);
++}
++
++SRes LzmaProps_Decode(CLzmaProps *p, const Byte *data, unsigned size)
++{
++ UInt32 dicSize;
++ Byte d;
++
++ if (size < LZMA_PROPS_SIZE)
++ return SZ_ERROR_UNSUPPORTED;
++ else
++ dicSize = data[1] | ((UInt32)data[2] << 8) | ((UInt32)data[3] << 16) | ((UInt32)data[4] << 24);
++
++ if (dicSize < LZMA_DIC_MIN)
++ dicSize = LZMA_DIC_MIN;
++ p->dicSize = dicSize;
++
++ d = data[0];
++ if (d >= (9 * 5 * 5))
++ return SZ_ERROR_UNSUPPORTED;
++
++ p->lc = d % 9;
++ d /= 9;
++ p->pb = d / 5;
++ p->lp = d % 5;
++
++ return SZ_OK;
++}
++
++static SRes LzmaDec_AllocateProbs2(CLzmaDec *p, const CLzmaProps *propNew, ISzAlloc *alloc)
++{
++ UInt32 numProbs = LzmaProps_GetNumProbs(propNew);
++ if (p->probs == 0 || numProbs != p->numProbs)
++ {
++ LzmaDec_FreeProbs(p, alloc);
++ p->probs = (CLzmaProb *)alloc->Alloc(alloc, numProbs * sizeof(CLzmaProb));
++ p->numProbs = numProbs;
++ if (p->probs == 0)
++ return SZ_ERROR_MEM;
++ }
++ return SZ_OK;
++}
++
++SRes LzmaDec_AllocateProbs(CLzmaDec *p, const Byte *props, unsigned propsSize, ISzAlloc *alloc)
++{
++ CLzmaProps propNew;
++ RINOK(LzmaProps_Decode(&propNew, props, propsSize));
++ RINOK(LzmaDec_AllocateProbs2(p, &propNew, alloc));
++ p->prop = propNew;
++ return SZ_OK;
++}
++
++SRes LzmaDec_Allocate(CLzmaDec *p, const Byte *props, unsigned propsSize, ISzAlloc *alloc)
++{
++ CLzmaProps propNew;
++ SizeT dicBufSize;
++ RINOK(LzmaProps_Decode(&propNew, props, propsSize));
++ RINOK(LzmaDec_AllocateProbs2(p, &propNew, alloc));
++ dicBufSize = propNew.dicSize;
++ if (p->dic == 0 || dicBufSize != p->dicBufSize)
++ {
++ LzmaDec_FreeDict(p, alloc);
++ p->dic = (Byte *)alloc->Alloc(alloc, dicBufSize);
++ if (p->dic == 0)
++ {
++ LzmaDec_FreeProbs(p, alloc);
++ return SZ_ERROR_MEM;
++ }
++ }
++ p->dicBufSize = dicBufSize;
++ p->prop = propNew;
++ return SZ_OK;
++}
++
++SRes LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
++ const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode,
++ ELzmaStatus *status, ISzAlloc *alloc)
++{
++ CLzmaDec p;
++ SRes res;
++ SizeT inSize = *srcLen;
++ SizeT outSize = *destLen;
++ *srcLen = *destLen = 0;
++ if (inSize < RC_INIT_SIZE)
++ return SZ_ERROR_INPUT_EOF;
++
++ LzmaDec_Construct(&p);
++ res = LzmaDec_AllocateProbs(&p, propData, propSize, alloc);
++ if (res != 0)
++ return res;
++ p.dic = dest;
++ p.dicBufSize = outSize;
++
++ LzmaDec_Init(&p);
++
++ *srcLen = inSize;
++ res = LzmaDec_DecodeToDic(&p, outSize, src, srcLen, finishMode, status);
++
++ if (res == SZ_OK && *status == LZMA_STATUS_NEEDS_MORE_INPUT)
++ res = SZ_ERROR_INPUT_EOF;
++
++ (*destLen) = p.dicPos;
++ LzmaDec_FreeProbs(&p, alloc);
++ return res;
++}
+
+Property changes on: third_party/lzma_sdk/LzmaDec.c
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/README.ots
+===================================================================
+--- third_party/lzma_sdk/README.ots (revision 0)
++++ third_party/lzma_sdk/README.ots (revision 0)
+@@ -0,0 +1,8 @@
++Name: LZMA SDK
++URL: http://www.7-zip.org/sdk.html
++Version: 9.20
++
++Description:
++The LZMA SDK provides the documentation, samples, header files, libraries, and tools you need to develop applications that use LZMA compression.
++
++This contains only the C code required to decompress LZMA.
+Index: third_party/lzma_sdk/lzma_sdk.gyp
+===================================================================
+--- third_party/lzma_sdk/lzma_sdk.gyp (revision 0)
++++ third_party/lzma_sdk/lzma_sdk.gyp (revision 0)
+@@ -0,0 +1,33 @@
++# Copyright (c) 2012 The Chromium Authors. All rights reserved.
++# Use of this source code is governed by a BSD-style license that can be
++# found in the LICENSE file.
++
++{
++ 'targets': [
++ {
++ 'target_name': 'lzma_sdk',
++ 'type': 'static_library',
++ 'defines': [
++ '_7ZIP_ST',
++ '_LZMA_PROB32',
++ ],
++ 'sources': [
++ 'Alloc.c',
++ 'Alloc.h',
++ 'LzFind.c',
++ 'LzFind.h',
++ 'LzHash.h',
++ 'LzmaEnc.c',
++ 'LzmaEnc.h',
++ 'LzmaDec.c',
++ 'LzmaDec.h',
++ 'LzmaLib.c',
++ 'LzmaLib.h',
++ 'Types.h',
++ ],
++ 'include_dirs': [
++ '.',
++ ],
++ },
++ ],
++}
+
+Property changes on: third_party/lzma_sdk/lzma_sdk.gyp
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzmaDec.h
+===================================================================
+--- third_party/lzma_sdk/LzmaDec.h (revision 0)
++++ third_party/lzma_sdk/LzmaDec.h (revision 0)
+@@ -0,0 +1,231 @@
++/* LzmaDec.h -- LZMA Decoder
++2009-02-07 : Igor Pavlov : Public domain */
++
++#ifndef __LZMA_DEC_H
++#define __LZMA_DEC_H
++
++#include "Types.h"
++
++#ifdef __cplusplus
++extern "C" {
++#endif
++
++/* #define _LZMA_PROB32 */
++/* _LZMA_PROB32 can increase the speed on some CPUs,
++ but memory usage for CLzmaDec::probs will be doubled in that case */
++
++#ifdef _LZMA_PROB32
++#define CLzmaProb UInt32
++#else
++#define CLzmaProb UInt16
++#endif
++
++
++/* ---------- LZMA Properties ---------- */
++
++#define LZMA_PROPS_SIZE 5
++
++typedef struct _CLzmaProps
++{
++ unsigned lc, lp, pb;
++ UInt32 dicSize;
++} CLzmaProps;
++
++/* LzmaProps_Decode - decodes properties
++Returns:
++ SZ_OK
++ SZ_ERROR_UNSUPPORTED - Unsupported properties
++*/
++
++SRes LzmaProps_Decode(CLzmaProps *p, const Byte *data, unsigned size);
++
++
++/* ---------- LZMA Decoder state ---------- */
++
++/* LZMA_REQUIRED_INPUT_MAX = number of required input bytes for worst case.
++ Num bits = log2((2^11 / 31) ^ 22) + 26 < 134 + 26 = 160; */
++
++#define LZMA_REQUIRED_INPUT_MAX 20
++
++typedef struct
++{
++ CLzmaProps prop;
++ CLzmaProb *probs;
++ Byte *dic;
++ const Byte *buf;
++ UInt32 range, code;
++ SizeT dicPos;
++ SizeT dicBufSize;
++ UInt32 processedPos;
++ UInt32 checkDicSize;
++ unsigned state;
++ UInt32 reps[4];
++ unsigned remainLen;
++ int needFlush;
++ int needInitState;
++ UInt32 numProbs;
++ unsigned tempBufSize;
++ Byte tempBuf[LZMA_REQUIRED_INPUT_MAX];
++} CLzmaDec;
++
++#define LzmaDec_Construct(p) { (p)->dic = 0; (p)->probs = 0; }
++
++void LzmaDec_Init(CLzmaDec *p);
++
++/* There are two types of LZMA streams:
++ 0) Stream with end mark. That end mark adds about 6 bytes to compressed size.
++ 1) Stream without end mark. You must know exact uncompressed size to decompress such stream. */
++
++typedef enum
++{
++ LZMA_FINISH_ANY, /* finish at any point */
++ LZMA_FINISH_END /* block must be finished at the end */
++} ELzmaFinishMode;
++
++/* ELzmaFinishMode has meaning only if the decoding reaches output limit !!!
++
++ You must use LZMA_FINISH_END, when you know that current output buffer
++ covers last bytes of block. In other cases you must use LZMA_FINISH_ANY.
++
++ If LZMA decoder sees end marker before reaching output limit, it returns SZ_OK,
++ and output value of destLen will be less than output buffer size limit.
++ You can check status result also.
++
++ You can use multiple checks to test data integrity after full decompression:
++ 1) Check Result and "status" variable.
++ 2) Check that output(destLen) = uncompressedSize, if you know real uncompressedSize.
++ 3) Check that output(srcLen) = compressedSize, if you know real compressedSize.
++ You must use correct finish mode in that case. */
++
++typedef enum
++{
++ LZMA_STATUS_NOT_SPECIFIED, /* use main error code instead */
++ LZMA_STATUS_FINISHED_WITH_MARK, /* stream was finished with end mark. */
++ LZMA_STATUS_NOT_FINISHED, /* stream was not finished */
++ LZMA_STATUS_NEEDS_MORE_INPUT, /* you must provide more input bytes */
++ LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK /* there is probability that stream was finished without end mark */
++} ELzmaStatus;
++
++/* ELzmaStatus is used only as output value for function call */
++
++
++/* ---------- Interfaces ---------- */
++
++/* There are 3 levels of interfaces:
++ 1) Dictionary Interface
++ 2) Buffer Interface
++ 3) One Call Interface
++ You can select any of these interfaces, but don't mix functions from different
++ groups for same object. */
++
++
++/* There are two variants to allocate state for Dictionary Interface:
++ 1) LzmaDec_Allocate / LzmaDec_Free
++ 2) LzmaDec_AllocateProbs / LzmaDec_FreeProbs
++ You can use variant 2, if you set dictionary buffer manually.
++ For Buffer Interface you must always use variant 1.
++
++LzmaDec_Allocate* can return:
++ SZ_OK
++ SZ_ERROR_MEM - Memory allocation error
++ SZ_ERROR_UNSUPPORTED - Unsupported properties
++*/
++
++SRes LzmaDec_AllocateProbs(CLzmaDec *p, const Byte *props, unsigned propsSize, ISzAlloc *alloc);
++void LzmaDec_FreeProbs(CLzmaDec *p, ISzAlloc *alloc);
++
++SRes LzmaDec_Allocate(CLzmaDec *state, const Byte *prop, unsigned propsSize, ISzAlloc *alloc);
++void LzmaDec_Free(CLzmaDec *state, ISzAlloc *alloc);
++
++/* ---------- Dictionary Interface ---------- */
++
++/* You can use it, if you want to eliminate the overhead for data copying from
++ dictionary to some other external buffer.
++ You must work with CLzmaDec variables directly in this interface.
++
++ STEPS:
++ LzmaDec_Constr()
++ LzmaDec_Allocate()
++ for (each new stream)
++ {
++ LzmaDec_Init()
++ while (it needs more decompression)
++ {
++ LzmaDec_DecodeToDic()
++ use data from CLzmaDec::dic and update CLzmaDec::dicPos
++ }
++ }
++ LzmaDec_Free()
++*/
++
++/* LzmaDec_DecodeToDic
++
++ The decoding to internal dictionary buffer (CLzmaDec::dic).
++ You must manually update CLzmaDec::dicPos, if it reaches CLzmaDec::dicBufSize !!!
++
++finishMode:
++ It has meaning only if the decoding reaches output limit (dicLimit).
++ LZMA_FINISH_ANY - Decode just dicLimit bytes.
++ LZMA_FINISH_END - Stream must be finished after dicLimit.
++
++Returns:
++ SZ_OK
++ status:
++ LZMA_STATUS_FINISHED_WITH_MARK
++ LZMA_STATUS_NOT_FINISHED
++ LZMA_STATUS_NEEDS_MORE_INPUT
++ LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK
++ SZ_ERROR_DATA - Data error
++*/
++
++SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit,
++ const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);
++
++
++/* ---------- Buffer Interface ---------- */
++
++/* It's zlib-like interface.
++ See LzmaDec_DecodeToDic description for information about STEPS and return results,
++ but you must use LzmaDec_DecodeToBuf instead of LzmaDec_DecodeToDic and you don't need
++ to work with CLzmaDec variables manually.
++
++finishMode:
++ It has meaning only if the decoding reaches output limit (*destLen).
++ LZMA_FINISH_ANY - Decode just destLen bytes.
++ LZMA_FINISH_END - Stream must be finished after (*destLen).
++*/
++
++SRes LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen,
++ const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);
++
++
++/* ---------- One Call Interface ---------- */
++
++/* LzmaDecode
++
++finishMode:
++ It has meaning only if the decoding reaches output limit (*destLen).
++ LZMA_FINISH_ANY - Decode just destLen bytes.
++ LZMA_FINISH_END - Stream must be finished after (*destLen).
++
++Returns:
++ SZ_OK
++ status:
++ LZMA_STATUS_FINISHED_WITH_MARK
++ LZMA_STATUS_NOT_FINISHED
++ LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK
++ SZ_ERROR_DATA - Data error
++ SZ_ERROR_MEM - Memory allocation error
++ SZ_ERROR_UNSUPPORTED - Unsupported properties
++ SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src).
++*/
++
++SRes LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
++ const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode,
++ ELzmaStatus *status, ISzAlloc *alloc);
++
++#ifdef __cplusplus
++}
++#endif
++
++#endif
+
+Property changes on: third_party/lzma_sdk/LzmaDec.h
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzFind.c
+===================================================================
+--- third_party/lzma_sdk/LzFind.c (revision 0)
++++ third_party/lzma_sdk/LzFind.c (revision 0)
+@@ -0,0 +1,761 @@
++/* LzFind.c -- Match finder for LZ algorithms
++2009-04-22 : Igor Pavlov : Public domain */
++
++#include <string.h>
++
++#include "LzFind.h"
++#include "LzHash.h"
++
++#define kEmptyHashValue 0
++#define kMaxValForNormalize ((UInt32)0xFFFFFFFF)
++#define kNormalizeStepMin (1 << 10) /* it must be power of 2 */
++#define kNormalizeMask (~(kNormalizeStepMin - 1))
++#define kMaxHistorySize ((UInt32)3 << 30)
++
++#define kStartMaxLen 3
++
++static void LzInWindow_Free(CMatchFinder *p, ISzAlloc *alloc)
++{
++ if (!p->directInput)
++ {
++ alloc->Free(alloc, p->bufferBase);
++ p->bufferBase = 0;
++ }
++}
++
++/* keepSizeBefore + keepSizeAfter + keepSizeReserv must be < 4G) */
++
++static int LzInWindow_Create(CMatchFinder *p, UInt32 keepSizeReserv, ISzAlloc *alloc)
++{
++ UInt32 blockSize = p->keepSizeBefore + p->keepSizeAfter + keepSizeReserv;
++ if (p->directInput)
++ {
++ p->blockSize = blockSize;
++ return 1;
++ }
++ if (p->bufferBase == 0 || p->blockSize != blockSize)
++ {
++ LzInWindow_Free(p, alloc);
++ p->blockSize = blockSize;
++ p->bufferBase = (Byte *)alloc->Alloc(alloc, (size_t)blockSize);
++ }
++ return (p->bufferBase != 0);
++}
++
++Byte *MatchFinder_GetPointerToCurrentPos(CMatchFinder *p) { return p->buffer; }
++Byte MatchFinder_GetIndexByte(CMatchFinder *p, Int32 index) { return p->buffer[index]; }
++
++UInt32 MatchFinder_GetNumAvailableBytes(CMatchFinder *p) { return p->streamPos - p->pos; }
++
++void MatchFinder_ReduceOffsets(CMatchFinder *p, UInt32 subValue)
++{
++ p->posLimit -= subValue;
++ p->pos -= subValue;
++ p->streamPos -= subValue;
++}
++
++static void MatchFinder_ReadBlock(CMatchFinder *p)
++{
++ if (p->streamEndWasReached || p->result != SZ_OK)
++ return;
++ if (p->directInput)
++ {
++ UInt32 curSize = 0xFFFFFFFF - p->streamPos;
++ if (curSize > p->directInputRem)
++ curSize = (UInt32)p->directInputRem;
++ p->directInputRem -= curSize;
++ p->streamPos += curSize;
++ if (p->directInputRem == 0)
++ p->streamEndWasReached = 1;
++ return;
++ }
++ for (;;)
++ {
++ Byte *dest = p->buffer + (p->streamPos - p->pos);
++ size_t size = (p->bufferBase + p->blockSize - dest);
++ if (size == 0)
++ return;
++ p->result = p->stream->Read(p->stream, dest, &size);
++ if (p->result != SZ_OK)
++ return;
++ if (size == 0)
++ {
++ p->streamEndWasReached = 1;
++ return;
++ }
++ p->streamPos += (UInt32)size;
++ if (p->streamPos - p->pos > p->keepSizeAfter)
++ return;
++ }
++}
++
++void MatchFinder_MoveBlock(CMatchFinder *p)
++{
++ memmove(p->bufferBase,
++ p->buffer - p->keepSizeBefore,
++ (size_t)(p->streamPos - p->pos + p->keepSizeBefore));
++ p->buffer = p->bufferBase + p->keepSizeBefore;
++}
++
++int MatchFinder_NeedMove(CMatchFinder *p)
++{
++ if (p->directInput)
++ return 0;
++ /* if (p->streamEndWasReached) return 0; */
++ return ((size_t)(p->bufferBase + p->blockSize - p->buffer) <= p->keepSizeAfter);
++}
++
++void MatchFinder_ReadIfRequired(CMatchFinder *p)
++{
++ if (p->streamEndWasReached)
++ return;
++ if (p->keepSizeAfter >= p->streamPos - p->pos)
++ MatchFinder_ReadBlock(p);
++}
++
++static void MatchFinder_CheckAndMoveAndRead(CMatchFinder *p)
++{
++ if (MatchFinder_NeedMove(p))
++ MatchFinder_MoveBlock(p);
++ MatchFinder_ReadBlock(p);
++}
++
++static void MatchFinder_SetDefaultSettings(CMatchFinder *p)
++{
++ p->cutValue = 32;
++ p->btMode = 1;
++ p->numHashBytes = 4;
++ p->bigHash = 0;
++}
++
++#define kCrcPoly 0xEDB88320
++
++void MatchFinder_Construct(CMatchFinder *p)
++{
++ UInt32 i;
++ p->bufferBase = 0;
++ p->directInput = 0;
++ p->hash = 0;
++ MatchFinder_SetDefaultSettings(p);
++
++ for (i = 0; i < 256; i++)
++ {
++ UInt32 r = i;
++ int j;
++ for (j = 0; j < 8; j++)
++ r = (r >> 1) ^ (kCrcPoly & ~((r & 1) - 1));
++ p->crc[i] = r;
++ }
++}
++
++static void MatchFinder_FreeThisClassMemory(CMatchFinder *p, ISzAlloc *alloc)
++{
++ alloc->Free(alloc, p->hash);
++ p->hash = 0;
++}
++
++void MatchFinder_Free(CMatchFinder *p, ISzAlloc *alloc)
++{
++ MatchFinder_FreeThisClassMemory(p, alloc);
++ LzInWindow_Free(p, alloc);
++}
++
++static CLzRef* AllocRefs(UInt32 num, ISzAlloc *alloc)
++{
++ size_t sizeInBytes = (size_t)num * sizeof(CLzRef);
++ if (sizeInBytes / sizeof(CLzRef) != num)
++ return 0;
++ return (CLzRef *)alloc->Alloc(alloc, sizeInBytes);
++}
++
++int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
++ UInt32 keepAddBufferBefore, UInt32 matchMaxLen, UInt32 keepAddBufferAfter,
++ ISzAlloc *alloc)
++{
++ UInt32 sizeReserv;
++ if (historySize > kMaxHistorySize)
++ {
++ MatchFinder_Free(p, alloc);
++ return 0;
++ }
++ sizeReserv = historySize >> 1;
++ if (historySize > ((UInt32)2 << 30))
++ sizeReserv = historySize >> 2;
++ sizeReserv += (keepAddBufferBefore + matchMaxLen + keepAddBufferAfter) / 2 + (1 << 19);
++
++ p->keepSizeBefore = historySize + keepAddBufferBefore + 1;
++ p->keepSizeAfter = matchMaxLen + keepAddBufferAfter;
++ /* we need one additional byte, since we use MoveBlock after pos++ and before dictionary using */
++ if (LzInWindow_Create(p, sizeReserv, alloc))
++ {
++ UInt32 newCyclicBufferSize = historySize + 1;
++ UInt32 hs;
++ p->matchMaxLen = matchMaxLen;
++ {
++ p->fixedHashSize = 0;
++ if (p->numHashBytes == 2)
++ hs = (1 << 16) - 1;
++ else
++ {
++ hs = historySize - 1;
++ hs |= (hs >> 1);
++ hs |= (hs >> 2);
++ hs |= (hs >> 4);
++ hs |= (hs >> 8);
++ hs >>= 1;
++ hs |= 0xFFFF; /* don't change it! It's required for Deflate */
++ if (hs > (1 << 24))
++ {
++ if (p->numHashBytes == 3)
++ hs = (1 << 24) - 1;
++ else
++ hs >>= 1;
++ }
++ }
++ p->hashMask = hs;
++ hs++;
++ if (p->numHashBytes > 2) p->fixedHashSize += kHash2Size;
++ if (p->numHashBytes > 3) p->fixedHashSize += kHash3Size;
++ if (p->numHashBytes > 4) p->fixedHashSize += kHash4Size;
++ hs += p->fixedHashSize;
++ }
++
++ {
++ UInt32 prevSize = p->hashSizeSum + p->numSons;
++ UInt32 newSize;
++ p->historySize = historySize;
++ p->hashSizeSum = hs;
++ p->cyclicBufferSize = newCyclicBufferSize;
++ p->numSons = (p->btMode ? newCyclicBufferSize * 2 : newCyclicBufferSize);
++ newSize = p->hashSizeSum + p->numSons;
++ if (p->hash != 0 && prevSize == newSize)
++ return 1;
++ MatchFinder_FreeThisClassMemory(p, alloc);
++ p->hash = AllocRefs(newSize, alloc);
++ if (p->hash != 0)
++ {
++ p->son = p->hash + p->hashSizeSum;
++ return 1;
++ }
++ }
++ }
++ MatchFinder_Free(p, alloc);
++ return 0;
++}
++
++static void MatchFinder_SetLimits(CMatchFinder *p)
++{
++ UInt32 limit = kMaxValForNormalize - p->pos;
++ UInt32 limit2 = p->cyclicBufferSize - p->cyclicBufferPos;
++ if (limit2 < limit)
++ limit = limit2;
++ limit2 = p->streamPos - p->pos;
++ if (limit2 <= p->keepSizeAfter)
++ {
++ if (limit2 > 0)
++ limit2 = 1;
++ }
++ else
++ limit2 -= p->keepSizeAfter;
++ if (limit2 < limit)
++ limit = limit2;
++ {
++ UInt32 lenLimit = p->streamPos - p->pos;
++ if (lenLimit > p->matchMaxLen)
++ lenLimit = p->matchMaxLen;
++ p->lenLimit = lenLimit;
++ }
++ p->posLimit = p->pos + limit;
++}
++
++void MatchFinder_Init(CMatchFinder *p)
++{
++ UInt32 i;
++ for (i = 0; i < p->hashSizeSum; i++)
++ p->hash[i] = kEmptyHashValue;
++ p->cyclicBufferPos = 0;
++ p->buffer = p->bufferBase;
++ p->pos = p->streamPos = p->cyclicBufferSize;
++ p->result = SZ_OK;
++ p->streamEndWasReached = 0;
++ MatchFinder_ReadBlock(p);
++ MatchFinder_SetLimits(p);
++}
++
++static UInt32 MatchFinder_GetSubValue(CMatchFinder *p)
++{
++ return (p->pos - p->historySize - 1) & kNormalizeMask;
++}
++
++void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, UInt32 numItems)
++{
++ UInt32 i;
++ for (i = 0; i < numItems; i++)
++ {
++ UInt32 value = items[i];
++ if (value <= subValue)
++ value = kEmptyHashValue;
++ else
++ value -= subValue;
++ items[i] = value;
++ }
++}
++
++static void MatchFinder_Normalize(CMatchFinder *p)
++{
++ UInt32 subValue = MatchFinder_GetSubValue(p);
++ MatchFinder_Normalize3(subValue, p->hash, p->hashSizeSum + p->numSons);
++ MatchFinder_ReduceOffsets(p, subValue);
++}
++
++static void MatchFinder_CheckLimits(CMatchFinder *p)
++{
++ if (p->pos == kMaxValForNormalize)
++ MatchFinder_Normalize(p);
++ if (!p->streamEndWasReached && p->keepSizeAfter == p->streamPos - p->pos)
++ MatchFinder_CheckAndMoveAndRead(p);
++ if (p->cyclicBufferPos == p->cyclicBufferSize)
++ p->cyclicBufferPos = 0;
++ MatchFinder_SetLimits(p);
++}
++
++static UInt32 * Hc_GetMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
++ UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
++ UInt32 *distances, UInt32 maxLen)
++{
++ son[_cyclicBufferPos] = curMatch;
++ for (;;)
++ {
++ UInt32 delta = pos - curMatch;
++ if (cutValue-- == 0 || delta >= _cyclicBufferSize)
++ return distances;
++ {
++ const Byte *pb = cur - delta;
++ curMatch = son[_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)];
++ if (pb[maxLen] == cur[maxLen] && *pb == *cur)
++ {
++ UInt32 len = 0;
++ while (++len != lenLimit)
++ if (pb[len] != cur[len])
++ break;
++ if (maxLen < len)
++ {
++ *distances++ = maxLen = len;
++ *distances++ = delta - 1;
++ if (len == lenLimit)
++ return distances;
++ }
++ }
++ }
++ }
++}
++
++UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
++ UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue,
++ UInt32 *distances, UInt32 maxLen)
++{
++ CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
++ CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
++ UInt32 len0 = 0, len1 = 0;
++ for (;;)
++ {
++ UInt32 delta = pos - curMatch;
++ if (cutValue-- == 0 || delta >= _cyclicBufferSize)
++ {
++ *ptr0 = *ptr1 = kEmptyHashValue;
++ return distances;
++ }
++ {
++ CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
++ const Byte *pb = cur - delta;
++ UInt32 len = (len0 < len1 ? len0 : len1);
++ if (pb[len] == cur[len])
++ {
++ if (++len != lenLimit && pb[len] == cur[len])
++ while (++len != lenLimit)
++ if (pb[len] != cur[len])
++ break;
++ if (maxLen < len)
++ {
++ *distances++ = maxLen = len;
++ *distances++ = delta - 1;
++ if (len == lenLimit)
++ {
++ *ptr1 = pair[0];
++ *ptr0 = pair[1];
++ return distances;
++ }
++ }
++ }
++ if (pb[len] < cur[len])
++ {
++ *ptr1 = curMatch;
++ ptr1 = pair + 1;
++ curMatch = *ptr1;
++ len1 = len;
++ }
++ else
++ {
++ *ptr0 = curMatch;
++ ptr0 = pair;
++ curMatch = *ptr0;
++ len0 = len;
++ }
++ }
++ }
++}
++
++static void SkipMatchesSpec(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *cur, CLzRef *son,
++ UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 cutValue)
++{
++ CLzRef *ptr0 = son + (_cyclicBufferPos << 1) + 1;
++ CLzRef *ptr1 = son + (_cyclicBufferPos << 1);
++ UInt32 len0 = 0, len1 = 0;
++ for (;;)
++ {
++ UInt32 delta = pos - curMatch;
++ if (cutValue-- == 0 || delta >= _cyclicBufferSize)
++ {
++ *ptr0 = *ptr1 = kEmptyHashValue;
++ return;
++ }
++ {
++ CLzRef *pair = son + ((_cyclicBufferPos - delta + ((delta > _cyclicBufferPos) ? _cyclicBufferSize : 0)) << 1);
++ const Byte *pb = cur - delta;
++ UInt32 len = (len0 < len1 ? len0 : len1);
++ if (pb[len] == cur[len])
++ {
++ while (++len != lenLimit)
++ if (pb[len] != cur[len])
++ break;
++ {
++ if (len == lenLimit)
++ {
++ *ptr1 = pair[0];
++ *ptr0 = pair[1];
++ return;
++ }
++ }
++ }
++ if (pb[len] < cur[len])
++ {
++ *ptr1 = curMatch;
++ ptr1 = pair + 1;
++ curMatch = *ptr1;
++ len1 = len;
++ }
++ else
++ {
++ *ptr0 = curMatch;
++ ptr0 = pair;
++ curMatch = *ptr0;
++ len0 = len;
++ }
++ }
++ }
++}
++
++#define MOVE_POS \
++ ++p->cyclicBufferPos; \
++ p->buffer++; \
++ if (++p->pos == p->posLimit) MatchFinder_CheckLimits(p);
++
++#define MOVE_POS_RET MOVE_POS return offset;
++
++static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; }
++
++#define GET_MATCHES_HEADER2(minLen, ret_op) \
++ UInt32 lenLimit; UInt32 hashValue; const Byte *cur; UInt32 curMatch; \
++ lenLimit = p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos(p); ret_op; }} \
++ cur = p->buffer;
++
++#define GET_MATCHES_HEADER(minLen) GET_MATCHES_HEADER2(minLen, return 0)
++#define SKIP_HEADER(minLen) GET_MATCHES_HEADER2(minLen, continue)
++
++#define MF_PARAMS(p) p->pos, p->buffer, p->son, p->cyclicBufferPos, p->cyclicBufferSize, p->cutValue
++
++#define GET_MATCHES_FOOTER(offset, maxLen) \
++ offset = (UInt32)(GetMatchesSpec1(lenLimit, curMatch, MF_PARAMS(p), \
++ distances + offset, maxLen) - distances); MOVE_POS_RET;
++
++#define SKIP_FOOTER \
++ SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS;
++
++static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
++{
++ UInt32 offset;
++ GET_MATCHES_HEADER(2)
++ HASH2_CALC;
++ curMatch = p->hash[hashValue];
++ p->hash[hashValue] = p->pos;
++ offset = 0;
++ GET_MATCHES_FOOTER(offset, 1)
++}
++
++UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
++{
++ UInt32 offset;
++ GET_MATCHES_HEADER(3)
++ HASH_ZIP_CALC;
++ curMatch = p->hash[hashValue];
++ p->hash[hashValue] = p->pos;
++ offset = 0;
++ GET_MATCHES_FOOTER(offset, 2)
++}
++
++static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
++{
++ UInt32 hash2Value, delta2, maxLen, offset;
++ GET_MATCHES_HEADER(3)
++
++ HASH3_CALC;
++
++ delta2 = p->pos - p->hash[hash2Value];
++ curMatch = p->hash[kFix3HashSize + hashValue];
++
++ p->hash[hash2Value] =
++ p->hash[kFix3HashSize + hashValue] = p->pos;
++
++
++ maxLen = 2;
++ offset = 0;
++ if (delta2 < p->cyclicBufferSize && *(cur - delta2) == *cur)
++ {
++ for (; maxLen != lenLimit; maxLen++)
++ if (cur[(ptrdiff_t)maxLen - delta2] != cur[maxLen])
++ break;
++ distances[0] = maxLen;
++ distances[1] = delta2 - 1;
++ offset = 2;
++ if (maxLen == lenLimit)
++ {
++ SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
++ MOVE_POS_RET;
++ }
++ }
++ GET_MATCHES_FOOTER(offset, maxLen)
++}
++
++static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
++{
++ UInt32 hash2Value, hash3Value, delta2, delta3, maxLen, offset;
++ GET_MATCHES_HEADER(4)
++
++ HASH4_CALC;
++
++ delta2 = p->pos - p->hash[ hash2Value];
++ delta3 = p->pos - p->hash[kFix3HashSize + hash3Value];
++ curMatch = p->hash[kFix4HashSize + hashValue];
++
++ p->hash[ hash2Value] =
++ p->hash[kFix3HashSize + hash3Value] =
++ p->hash[kFix4HashSize + hashValue] = p->pos;
++
++ maxLen = 1;
++ offset = 0;
++ if (delta2 < p->cyclicBufferSize && *(cur - delta2) == *cur)
++ {
++ distances[0] = maxLen = 2;
++ distances[1] = delta2 - 1;
++ offset = 2;
++ }
++ if (delta2 != delta3 && delta3 < p->cyclicBufferSize && *(cur - delta3) == *cur)
++ {
++ maxLen = 3;
++ distances[offset + 1] = delta3 - 1;
++ offset += 2;
++ delta2 = delta3;
++ }
++ if (offset != 0)
++ {
++ for (; maxLen != lenLimit; maxLen++)
++ if (cur[(ptrdiff_t)maxLen - delta2] != cur[maxLen])
++ break;
++ distances[offset - 2] = maxLen;
++ if (maxLen == lenLimit)
++ {
++ SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p));
++ MOVE_POS_RET;
++ }
++ }
++ if (maxLen < 3)
++ maxLen = 3;
++ GET_MATCHES_FOOTER(offset, maxLen)
++}
++
++static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
++{
++ UInt32 hash2Value, hash3Value, delta2, delta3, maxLen, offset;
++ GET_MATCHES_HEADER(4)
++
++ HASH4_CALC;
++
++ delta2 = p->pos - p->hash[ hash2Value];
++ delta3 = p->pos - p->hash[kFix3HashSize + hash3Value];
++ curMatch = p->hash[kFix4HashSize + hashValue];
++
++ p->hash[ hash2Value] =
++ p->hash[kFix3HashSize + hash3Value] =
++ p->hash[kFix4HashSize + hashValue] = p->pos;
++
++ maxLen = 1;
++ offset = 0;
++ if (delta2 < p->cyclicBufferSize && *(cur - delta2) == *cur)
++ {
++ distances[0] = maxLen = 2;
++ distances[1] = delta2 - 1;
++ offset = 2;
++ }
++ if (delta2 != delta3 && delta3 < p->cyclicBufferSize && *(cur - delta3) == *cur)
++ {
++ maxLen = 3;
++ distances[offset + 1] = delta3 - 1;
++ offset += 2;
++ delta2 = delta3;
++ }
++ if (offset != 0)
++ {
++ for (; maxLen != lenLimit; maxLen++)
++ if (cur[(ptrdiff_t)maxLen - delta2] != cur[maxLen])
++ break;
++ distances[offset - 2] = maxLen;
++ if (maxLen == lenLimit)
++ {
++ p->son[p->cyclicBufferPos] = curMatch;
++ MOVE_POS_RET;
++ }
++ }
++ if (maxLen < 3)
++ maxLen = 3;
++ offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
++ distances + offset, maxLen) - (distances));
++ MOVE_POS_RET
++}
++
++UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances)
++{
++ UInt32 offset;
++ GET_MATCHES_HEADER(3)
++ HASH_ZIP_CALC;
++ curMatch = p->hash[hashValue];
++ p->hash[hashValue] = p->pos;
++ offset = (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p),
++ distances, 2) - (distances));
++ MOVE_POS_RET
++}
++
++static void Bt2_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
++{
++ do
++ {
++ SKIP_HEADER(2)
++ HASH2_CALC;
++ curMatch = p->hash[hashValue];
++ p->hash[hashValue] = p->pos;
++ SKIP_FOOTER
++ }
++ while (--num != 0);
++}
++
++void Bt3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
++{
++ do
++ {
++ SKIP_HEADER(3)
++ HASH_ZIP_CALC;
++ curMatch = p->hash[hashValue];
++ p->hash[hashValue] = p->pos;
++ SKIP_FOOTER
++ }
++ while (--num != 0);
++}
++
++static void Bt3_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
++{
++ do
++ {
++ UInt32 hash2Value;
++ SKIP_HEADER(3)
++ HASH3_CALC;
++ curMatch = p->hash[kFix3HashSize + hashValue];
++ p->hash[hash2Value] =
++ p->hash[kFix3HashSize + hashValue] = p->pos;
++ SKIP_FOOTER
++ }
++ while (--num != 0);
++}
++
++static void Bt4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
++{
++ do
++ {
++ UInt32 hash2Value, hash3Value;
++ SKIP_HEADER(4)
++ HASH4_CALC;
++ curMatch = p->hash[kFix4HashSize + hashValue];
++ p->hash[ hash2Value] =
++ p->hash[kFix3HashSize + hash3Value] = p->pos;
++ p->hash[kFix4HashSize + hashValue] = p->pos;
++ SKIP_FOOTER
++ }
++ while (--num != 0);
++}
++
++static void Hc4_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
++{
++ do
++ {
++ UInt32 hash2Value, hash3Value;
++ SKIP_HEADER(4)
++ HASH4_CALC;
++ curMatch = p->hash[kFix4HashSize + hashValue];
++ p->hash[ hash2Value] =
++ p->hash[kFix3HashSize + hash3Value] =
++ p->hash[kFix4HashSize + hashValue] = p->pos;
++ p->son[p->cyclicBufferPos] = curMatch;
++ MOVE_POS
++ }
++ while (--num != 0);
++}
++
++void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num)
++{
++ do
++ {
++ SKIP_HEADER(3)
++ HASH_ZIP_CALC;
++ curMatch = p->hash[hashValue];
++ p->hash[hashValue] = p->pos;
++ p->son[p->cyclicBufferPos] = curMatch;
++ MOVE_POS
++ }
++ while (--num != 0);
++}
++
++void MatchFinder_CreateVTable(CMatchFinder *p, IMatchFinder *vTable)
++{
++ vTable->Init = (Mf_Init_Func)MatchFinder_Init;
++ vTable->GetIndexByte = (Mf_GetIndexByte_Func)MatchFinder_GetIndexByte;
++ vTable->GetNumAvailableBytes = (Mf_GetNumAvailableBytes_Func)MatchFinder_GetNumAvailableBytes;
++ vTable->GetPointerToCurrentPos = (Mf_GetPointerToCurrentPos_Func)MatchFinder_GetPointerToCurrentPos;
++ if (!p->btMode)
++ {
++ vTable->GetMatches = (Mf_GetMatches_Func)Hc4_MatchFinder_GetMatches;
++ vTable->Skip = (Mf_Skip_Func)Hc4_MatchFinder_Skip;
++ }
++ else if (p->numHashBytes == 2)
++ {
++ vTable->GetMatches = (Mf_GetMatches_Func)Bt2_MatchFinder_GetMatches;
++ vTable->Skip = (Mf_Skip_Func)Bt2_MatchFinder_Skip;
++ }
++ else if (p->numHashBytes == 3)
++ {
++ vTable->GetMatches = (Mf_GetMatches_Func)Bt3_MatchFinder_GetMatches;
++ vTable->Skip = (Mf_Skip_Func)Bt3_MatchFinder_Skip;
++ }
++ else
++ {
++ vTable->GetMatches = (Mf_GetMatches_Func)Bt4_MatchFinder_GetMatches;
++ vTable->Skip = (Mf_Skip_Func)Bt4_MatchFinder_Skip;
++ }
++}
+
+Property changes on: third_party/lzma_sdk/LzFind.c
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzmaEnc.c
+===================================================================
+--- third_party/lzma_sdk/LzmaEnc.c (revision 0)
++++ third_party/lzma_sdk/LzmaEnc.c (revision 0)
+@@ -0,0 +1,2268 @@
++/* LzmaEnc.c -- LZMA Encoder
++2010-04-16 : Igor Pavlov : Public domain */
++
++#include <string.h>
++
++/* #define SHOW_STAT */
++/* #define SHOW_STAT2 */
++
++#if defined(SHOW_STAT) || defined(SHOW_STAT2)
++#include <stdio.h>
++#endif
++
++#include "LzmaEnc.h"
++
++#include "LzFind.h"
++#ifndef _7ZIP_ST
++#include "LzFindMt.h"
++#endif
++
++#ifdef SHOW_STAT
++static int ttt = 0;
++#endif
++
++#define kBlockSizeMax ((1 << LZMA_NUM_BLOCK_SIZE_BITS) - 1)
++
++#define kBlockSize (9 << 10)
++#define kUnpackBlockSize (1 << 18)
++#define kMatchArraySize (1 << 21)
++#define kMatchRecordMaxSize ((LZMA_MATCH_LEN_MAX * 2 + 3) * LZMA_MATCH_LEN_MAX)
++
++#define kNumMaxDirectBits (31)
++
++#define kNumTopBits 24
++#define kTopValue ((UInt32)1 << kNumTopBits)
++
++#define kNumBitModelTotalBits 11
++#define kBitModelTotal (1 << kNumBitModelTotalBits)
++#define kNumMoveBits 5
++#define kProbInitValue (kBitModelTotal >> 1)
++
++#define kNumMoveReducingBits 4
++#define kNumBitPriceShiftBits 4
++#define kBitPrice (1 << kNumBitPriceShiftBits)
++
++void LzmaEncProps_Init(CLzmaEncProps *p)
++{
++ p->level = 5;
++ p->dictSize = p->mc = 0;
++ p->lc = p->lp = p->pb = p->algo = p->fb = p->btMode = p->numHashBytes = p->numThreads = -1;
++ p->writeEndMark = 0;
++}
++
++void LzmaEncProps_Normalize(CLzmaEncProps *p)
++{
++ int level = p->level;
++ if (level < 0) level = 5;
++ p->level = level;
++ if (p->dictSize == 0) p->dictSize = (level <= 5 ? (1 << (level * 2 + 14)) : (level == 6 ? (1 << 25) : (1 << 26)));
++ if (p->lc < 0) p->lc = 3;
++ if (p->lp < 0) p->lp = 0;
++ if (p->pb < 0) p->pb = 2;
++ if (p->algo < 0) p->algo = (level < 5 ? 0 : 1);
++ if (p->fb < 0) p->fb = (level < 7 ? 32 : 64);
++ if (p->btMode < 0) p->btMode = (p->algo == 0 ? 0 : 1);
++ if (p->numHashBytes < 0) p->numHashBytes = 4;
++ if (p->mc == 0) p->mc = (16 + (p->fb >> 1)) >> (p->btMode ? 0 : 1);
++ if (p->numThreads < 0)
++ p->numThreads =
++ #ifndef _7ZIP_ST
++ ((p->btMode && p->algo) ? 2 : 1);
++ #else
++ 1;
++ #endif
++}
++
++UInt32 LzmaEncProps_GetDictSize(const CLzmaEncProps *props2)
++{
++ CLzmaEncProps props = *props2;
++ LzmaEncProps_Normalize(&props);
++ return props.dictSize;
++}
++
++/* #define LZMA_LOG_BSR */
++/* Define it for Intel's CPU */
++
++
++#ifdef LZMA_LOG_BSR
++
++#define kDicLogSizeMaxCompress 30
++
++#define BSR2_RET(pos, res) { unsigned long i; _BitScanReverse(&i, (pos)); res = (i + i) + ((pos >> (i - 1)) & 1); }
++
++UInt32 GetPosSlot1(UInt32 pos)
++{
++ UInt32 res;
++ BSR2_RET(pos, res);
++ return res;
++}
++#define GetPosSlot2(pos, res) { BSR2_RET(pos, res); }
++#define GetPosSlot(pos, res) { if (pos < 2) res = pos; else BSR2_RET(pos, res); }
++
++#else
++
++#define kNumLogBits (9 + (int)sizeof(size_t) / 2)
++#define kDicLogSizeMaxCompress ((kNumLogBits - 1) * 2 + 7)
++
++void LzmaEnc_FastPosInit(Byte *g_FastPos)
++{
++ int c = 2, slotFast;
++ g_FastPos[0] = 0;
++ g_FastPos[1] = 1;
++
++ for (slotFast = 2; slotFast < kNumLogBits * 2; slotFast++)
++ {
++ UInt32 k = (1 << ((slotFast >> 1) - 1));
++ UInt32 j;
++ for (j = 0; j < k; j++, c++)
++ g_FastPos[c] = (Byte)slotFast;
++ }
++}
++
++#define BSR2_RET(pos, res) { UInt32 i = 6 + ((kNumLogBits - 1) & \
++ (0 - (((((UInt32)1 << (kNumLogBits + 6)) - 1) - pos) >> 31))); \
++ res = p->g_FastPos[pos >> i] + (i * 2); }
++/*
++#define BSR2_RET(pos, res) { res = (pos < (1 << (kNumLogBits + 6))) ? \
++ p->g_FastPos[pos >> 6] + 12 : \
++ p->g_FastPos[pos >> (6 + kNumLogBits - 1)] + (6 + (kNumLogBits - 1)) * 2; }
++*/
++
++#define GetPosSlot1(pos) p->g_FastPos[pos]
++#define GetPosSlot2(pos, res) { BSR2_RET(pos, res); }
++#define GetPosSlot(pos, res) { if (pos < kNumFullDistances) res = p->g_FastPos[pos]; else BSR2_RET(pos, res); }
++
++#endif
++
++
++#define LZMA_NUM_REPS 4
++
++typedef unsigned CState;
++
++typedef struct
++{
++ UInt32 price;
++
++ CState state;
++ int prev1IsChar;
++ int prev2;
++
++ UInt32 posPrev2;
++ UInt32 backPrev2;
++
++ UInt32 posPrev;
++ UInt32 backPrev;
++ UInt32 backs[LZMA_NUM_REPS];
++} COptimal;
++
++#define kNumOpts (1 << 12)
++
++#define kNumLenToPosStates 4
++#define kNumPosSlotBits 6
++#define kDicLogSizeMin 0
++#define kDicLogSizeMax 32
++#define kDistTableSizeMax (kDicLogSizeMax * 2)
++
++
++#define kNumAlignBits 4
++#define kAlignTableSize (1 << kNumAlignBits)
++#define kAlignMask (kAlignTableSize - 1)
++
++#define kStartPosModelIndex 4
++#define kEndPosModelIndex 14
++#define kNumPosModels (kEndPosModelIndex - kStartPosModelIndex)
++
++#define kNumFullDistances (1 << (kEndPosModelIndex >> 1))
++
++#ifdef _LZMA_PROB32
++#define CLzmaProb UInt32
++#else
++#define CLzmaProb UInt16
++#endif
++
++#define LZMA_PB_MAX 4
++#define LZMA_LC_MAX 8
++#define LZMA_LP_MAX 4
++
++#define LZMA_NUM_PB_STATES_MAX (1 << LZMA_PB_MAX)
++
++
++#define kLenNumLowBits 3
++#define kLenNumLowSymbols (1 << kLenNumLowBits)
++#define kLenNumMidBits 3
++#define kLenNumMidSymbols (1 << kLenNumMidBits)
++#define kLenNumHighBits 8
++#define kLenNumHighSymbols (1 << kLenNumHighBits)
++
++#define kLenNumSymbolsTotal (kLenNumLowSymbols + kLenNumMidSymbols + kLenNumHighSymbols)
++
++#define LZMA_MATCH_LEN_MIN 2
++#define LZMA_MATCH_LEN_MAX (LZMA_MATCH_LEN_MIN + kLenNumSymbolsTotal - 1)
++
++#define kNumStates 12
++
++typedef struct
++{
++ CLzmaProb choice;
++ CLzmaProb choice2;
++ CLzmaProb low[LZMA_NUM_PB_STATES_MAX << kLenNumLowBits];
++ CLzmaProb mid[LZMA_NUM_PB_STATES_MAX << kLenNumMidBits];
++ CLzmaProb high[kLenNumHighSymbols];
++} CLenEnc;
++
++typedef struct
++{
++ CLenEnc p;
++ UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal];
++ UInt32 tableSize;
++ UInt32 counters[LZMA_NUM_PB_STATES_MAX];
++} CLenPriceEnc;
++
++typedef struct
++{
++ UInt32 range;
++ Byte cache;
++ UInt64 low;
++ UInt64 cacheSize;
++ Byte *buf;
++ Byte *bufLim;
++ Byte *bufBase;
++ ISeqOutStream *outStream;
++ UInt64 processed;
++ SRes res;
++} CRangeEnc;
++
++typedef struct
++{
++ CLzmaProb *litProbs;
++
++ CLzmaProb isMatch[kNumStates][LZMA_NUM_PB_STATES_MAX];
++ CLzmaProb isRep[kNumStates];
++ CLzmaProb isRepG0[kNumStates];
++ CLzmaProb isRepG1[kNumStates];
++ CLzmaProb isRepG2[kNumStates];
++ CLzmaProb isRep0Long[kNumStates][LZMA_NUM_PB_STATES_MAX];
++
++ CLzmaProb posSlotEncoder[kNumLenToPosStates][1 << kNumPosSlotBits];
++ CLzmaProb posEncoders[kNumFullDistances - kEndPosModelIndex];
++ CLzmaProb posAlignEncoder[1 << kNumAlignBits];
++
++ CLenPriceEnc lenEnc;
++ CLenPriceEnc repLenEnc;
++
++ UInt32 reps[LZMA_NUM_REPS];
++ UInt32 state;
++} CSaveState;
++
++typedef struct
++{
++ IMatchFinder matchFinder;
++ void *matchFinderObj;
++
++ #ifndef _7ZIP_ST
++ Bool mtMode;
++ CMatchFinderMt matchFinderMt;
++ #endif
++
++ CMatchFinder matchFinderBase;
++
++ #ifndef _7ZIP_ST
++ Byte pad[128];
++ #endif
++
++ UInt32 optimumEndIndex;
++ UInt32 optimumCurrentIndex;
++
++ UInt32 longestMatchLength;
++ UInt32 numPairs;
++ UInt32 numAvail;
++ COptimal opt[kNumOpts];
++
++ #ifndef LZMA_LOG_BSR
++ Byte g_FastPos[1 << kNumLogBits];
++ #endif
++
++ UInt32 ProbPrices[kBitModelTotal >> kNumMoveReducingBits];
++ UInt32 matches[LZMA_MATCH_LEN_MAX * 2 + 2 + 1];
++ UInt32 numFastBytes;
++ UInt32 additionalOffset;
++ UInt32 reps[LZMA_NUM_REPS];
++ UInt32 state;
++
++ UInt32 posSlotPrices[kNumLenToPosStates][kDistTableSizeMax];
++ UInt32 distancesPrices[kNumLenToPosStates][kNumFullDistances];
++ UInt32 alignPrices[kAlignTableSize];
++ UInt32 alignPriceCount;
++
++ UInt32 distTableSize;
++
++ unsigned lc, lp, pb;
++ unsigned lpMask, pbMask;
++
++ CLzmaProb *litProbs;
++
++ CLzmaProb isMatch[kNumStates][LZMA_NUM_PB_STATES_MAX];
++ CLzmaProb isRep[kNumStates];
++ CLzmaProb isRepG0[kNumStates];
++ CLzmaProb isRepG1[kNumStates];
++ CLzmaProb isRepG2[kNumStates];
++ CLzmaProb isRep0Long[kNumStates][LZMA_NUM_PB_STATES_MAX];
++
++ CLzmaProb posSlotEncoder[kNumLenToPosStates][1 << kNumPosSlotBits];
++ CLzmaProb posEncoders[kNumFullDistances - kEndPosModelIndex];
++ CLzmaProb posAlignEncoder[1 << kNumAlignBits];
++
++ CLenPriceEnc lenEnc;
++ CLenPriceEnc repLenEnc;
++
++ unsigned lclp;
++
++ Bool fastMode;
++
++ CRangeEnc rc;
++
++ Bool writeEndMark;
++ UInt64 nowPos64;
++ UInt32 matchPriceCount;
++ Bool finished;
++ Bool multiThread;
++
++ SRes result;
++ UInt32 dictSize;
++ UInt32 matchFinderCycles;
++
++ int needInit;
++
++ CSaveState saveState;
++} CLzmaEnc;
++
++void LzmaEnc_SaveState(CLzmaEncHandle pp)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ CSaveState *dest = &p->saveState;
++ int i;
++ dest->lenEnc = p->lenEnc;
++ dest->repLenEnc = p->repLenEnc;
++ dest->state = p->state;
++
++ for (i = 0; i < kNumStates; i++)
++ {
++ memcpy(dest->isMatch[i], p->isMatch[i], sizeof(p->isMatch[i]));
++ memcpy(dest->isRep0Long[i], p->isRep0Long[i], sizeof(p->isRep0Long[i]));
++ }
++ for (i = 0; i < kNumLenToPosStates; i++)
++ memcpy(dest->posSlotEncoder[i], p->posSlotEncoder[i], sizeof(p->posSlotEncoder[i]));
++ memcpy(dest->isRep, p->isRep, sizeof(p->isRep));
++ memcpy(dest->isRepG0, p->isRepG0, sizeof(p->isRepG0));
++ memcpy(dest->isRepG1, p->isRepG1, sizeof(p->isRepG1));
++ memcpy(dest->isRepG2, p->isRepG2, sizeof(p->isRepG2));
++ memcpy(dest->posEncoders, p->posEncoders, sizeof(p->posEncoders));
++ memcpy(dest->posAlignEncoder, p->posAlignEncoder, sizeof(p->posAlignEncoder));
++ memcpy(dest->reps, p->reps, sizeof(p->reps));
++ memcpy(dest->litProbs, p->litProbs, (0x300 << p->lclp) * sizeof(CLzmaProb));
++}
++
++void LzmaEnc_RestoreState(CLzmaEncHandle pp)
++{
++ CLzmaEnc *dest = (CLzmaEnc *)pp;
++ const CSaveState *p = &dest->saveState;
++ int i;
++ dest->lenEnc = p->lenEnc;
++ dest->repLenEnc = p->repLenEnc;
++ dest->state = p->state;
++
++ for (i = 0; i < kNumStates; i++)
++ {
++ memcpy(dest->isMatch[i], p->isMatch[i], sizeof(p->isMatch[i]));
++ memcpy(dest->isRep0Long[i], p->isRep0Long[i], sizeof(p->isRep0Long[i]));
++ }
++ for (i = 0; i < kNumLenToPosStates; i++)
++ memcpy(dest->posSlotEncoder[i], p->posSlotEncoder[i], sizeof(p->posSlotEncoder[i]));
++ memcpy(dest->isRep, p->isRep, sizeof(p->isRep));
++ memcpy(dest->isRepG0, p->isRepG0, sizeof(p->isRepG0));
++ memcpy(dest->isRepG1, p->isRepG1, sizeof(p->isRepG1));
++ memcpy(dest->isRepG2, p->isRepG2, sizeof(p->isRepG2));
++ memcpy(dest->posEncoders, p->posEncoders, sizeof(p->posEncoders));
++ memcpy(dest->posAlignEncoder, p->posAlignEncoder, sizeof(p->posAlignEncoder));
++ memcpy(dest->reps, p->reps, sizeof(p->reps));
++ memcpy(dest->litProbs, p->litProbs, (0x300 << dest->lclp) * sizeof(CLzmaProb));
++}
++
++SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEncProps *props2)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ CLzmaEncProps props = *props2;
++ LzmaEncProps_Normalize(&props);
++
++ if (props.lc > LZMA_LC_MAX || props.lp > LZMA_LP_MAX || props.pb > LZMA_PB_MAX ||
++ props.dictSize > ((UInt32)1 << kDicLogSizeMaxCompress) || props.dictSize > ((UInt32)1 << 30))
++ return SZ_ERROR_PARAM;
++ p->dictSize = props.dictSize;
++ p->matchFinderCycles = props.mc;
++ {
++ unsigned fb = props.fb;
++ if (fb < 5)
++ fb = 5;
++ if (fb > LZMA_MATCH_LEN_MAX)
++ fb = LZMA_MATCH_LEN_MAX;
++ p->numFastBytes = fb;
++ }
++ p->lc = props.lc;
++ p->lp = props.lp;
++ p->pb = props.pb;
++ p->fastMode = (props.algo == 0);
++ p->matchFinderBase.btMode = props.btMode;
++ {
++ UInt32 numHashBytes = 4;
++ if (props.btMode)
++ {
++ if (props.numHashBytes < 2)
++ numHashBytes = 2;
++ else if (props.numHashBytes < 4)
++ numHashBytes = props.numHashBytes;
++ }
++ p->matchFinderBase.numHashBytes = numHashBytes;
++ }
++
++ p->matchFinderBase.cutValue = props.mc;
++
++ p->writeEndMark = props.writeEndMark;
++
++ #ifndef _7ZIP_ST
++ /*
++ if (newMultiThread != _multiThread)
++ {
++ ReleaseMatchFinder();
++ _multiThread = newMultiThread;
++ }
++ */
++ p->multiThread = (props.numThreads > 1);
++ #endif
++
++ return SZ_OK;
++}
++
++static const int kLiteralNextStates[kNumStates] = {0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5};
++static const int kMatchNextStates[kNumStates] = {7, 7, 7, 7, 7, 7, 7, 10, 10, 10, 10, 10};
++static const int kRepNextStates[kNumStates] = {8, 8, 8, 8, 8, 8, 8, 11, 11, 11, 11, 11};
++static const int kShortRepNextStates[kNumStates]= {9, 9, 9, 9, 9, 9, 9, 11, 11, 11, 11, 11};
++
++#define IsCharState(s) ((s) < 7)
++
++#define GetLenToPosState(len) (((len) < kNumLenToPosStates + 1) ? (len) - 2 : kNumLenToPosStates - 1)
++
++#define kInfinityPrice (1 << 30)
++
++static void RangeEnc_Construct(CRangeEnc *p)
++{
++ p->outStream = 0;
++ p->bufBase = 0;
++}
++
++#define RangeEnc_GetProcessed(p) ((p)->processed + ((p)->buf - (p)->bufBase) + (p)->cacheSize)
++
++#define RC_BUF_SIZE (1 << 16)
++static int RangeEnc_Alloc(CRangeEnc *p, ISzAlloc *alloc)
++{
++ if (p->bufBase == 0)
++ {
++ p->bufBase = (Byte *)alloc->Alloc(alloc, RC_BUF_SIZE);
++ if (p->bufBase == 0)
++ return 0;
++ p->bufLim = p->bufBase + RC_BUF_SIZE;
++ }
++ return 1;
++}
++
++static void RangeEnc_Free(CRangeEnc *p, ISzAlloc *alloc)
++{
++ alloc->Free(alloc, p->bufBase);
++ p->bufBase = 0;
++}
++
++static void RangeEnc_Init(CRangeEnc *p)
++{
++ /* Stream.Init(); */
++ p->low = 0;
++ p->range = 0xFFFFFFFF;
++ p->cacheSize = 1;
++ p->cache = 0;
++
++ p->buf = p->bufBase;
++
++ p->processed = 0;
++ p->res = SZ_OK;
++}
++
++static void RangeEnc_FlushStream(CRangeEnc *p)
++{
++ size_t num;
++ if (p->res != SZ_OK)
++ return;
++ num = p->buf - p->bufBase;
++ if (num != p->outStream->Write(p->outStream, p->bufBase, num))
++ p->res = SZ_ERROR_WRITE;
++ p->processed += num;
++ p->buf = p->bufBase;
++}
++
++static void MY_FAST_CALL RangeEnc_ShiftLow(CRangeEnc *p)
++{
++ if ((UInt32)p->low < (UInt32)0xFF000000 || (int)(p->low >> 32) != 0)
++ {
++ Byte temp = p->cache;
++ do
++ {
++ Byte *buf = p->buf;
++ *buf++ = (Byte)(temp + (Byte)(p->low >> 32));
++ p->buf = buf;
++ if (buf == p->bufLim)
++ RangeEnc_FlushStream(p);
++ temp = 0xFF;
++ }
++ while (--p->cacheSize != 0);
++ p->cache = (Byte)((UInt32)p->low >> 24);
++ }
++ p->cacheSize++;
++ p->low = (UInt32)p->low << 8;
++}
++
++static void RangeEnc_FlushData(CRangeEnc *p)
++{
++ int i;
++ for (i = 0; i < 5; i++)
++ RangeEnc_ShiftLow(p);
++}
++
++static void RangeEnc_EncodeDirectBits(CRangeEnc *p, UInt32 value, int numBits)
++{
++ do
++ {
++ p->range >>= 1;
++ p->low += p->range & (0 - ((value >> --numBits) & 1));
++ if (p->range < kTopValue)
++ {
++ p->range <<= 8;
++ RangeEnc_ShiftLow(p);
++ }
++ }
++ while (numBits != 0);
++}
++
++static void RangeEnc_EncodeBit(CRangeEnc *p, CLzmaProb *prob, UInt32 symbol)
++{
++ UInt32 ttt = *prob;
++ UInt32 newBound = (p->range >> kNumBitModelTotalBits) * ttt;
++ if (symbol == 0)
++ {
++ p->range = newBound;
++ ttt += (kBitModelTotal - ttt) >> kNumMoveBits;
++ }
++ else
++ {
++ p->low += newBound;
++ p->range -= newBound;
++ ttt -= ttt >> kNumMoveBits;
++ }
++ *prob = (CLzmaProb)ttt;
++ if (p->range < kTopValue)
++ {
++ p->range <<= 8;
++ RangeEnc_ShiftLow(p);
++ }
++}
++
++static void LitEnc_Encode(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol)
++{
++ symbol |= 0x100;
++ do
++ {
++ RangeEnc_EncodeBit(p, probs + (symbol >> 8), (symbol >> 7) & 1);
++ symbol <<= 1;
++ }
++ while (symbol < 0x10000);
++}
++
++static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaProb *probs, UInt32 symbol, UInt32 matchByte)
++{
++ UInt32 offs = 0x100;
++ symbol |= 0x100;
++ do
++ {
++ matchByte <<= 1;
++ RangeEnc_EncodeBit(p, probs + (offs + (matchByte & offs) + (symbol >> 8)), (symbol >> 7) & 1);
++ symbol <<= 1;
++ offs &= ~(matchByte ^ symbol);
++ }
++ while (symbol < 0x10000);
++}
++
++void LzmaEnc_InitPriceTables(UInt32 *ProbPrices)
++{
++ UInt32 i;
++ for (i = (1 << kNumMoveReducingBits) / 2; i < kBitModelTotal; i += (1 << kNumMoveReducingBits))
++ {
++ const int kCyclesBits = kNumBitPriceShiftBits;
++ UInt32 w = i;
++ UInt32 bitCount = 0;
++ int j;
++ for (j = 0; j < kCyclesBits; j++)
++ {
++ w = w * w;
++ bitCount <<= 1;
++ while (w >= ((UInt32)1 << 16))
++ {
++ w >>= 1;
++ bitCount++;
++ }
++ }
++ ProbPrices[i >> kNumMoveReducingBits] = ((kNumBitModelTotalBits << kCyclesBits) - 15 - bitCount);
++ }
++}
++
++
++#define GET_PRICE(prob, symbol) \
++ p->ProbPrices[((prob) ^ (((-(int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
++
++#define GET_PRICEa(prob, symbol) \
++ ProbPrices[((prob) ^ ((-((int)(symbol))) & (kBitModelTotal - 1))) >> kNumMoveReducingBits];
++
++#define GET_PRICE_0(prob) p->ProbPrices[(prob) >> kNumMoveReducingBits]
++#define GET_PRICE_1(prob) p->ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
++
++#define GET_PRICE_0a(prob) ProbPrices[(prob) >> kNumMoveReducingBits]
++#define GET_PRICE_1a(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> kNumMoveReducingBits]
++
++static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol, UInt32 *ProbPrices)
++{
++ UInt32 price = 0;
++ symbol |= 0x100;
++ do
++ {
++ price += GET_PRICEa(probs[symbol >> 8], (symbol >> 7) & 1);
++ symbol <<= 1;
++ }
++ while (symbol < 0x10000);
++ return price;
++}
++
++static UInt32 LitEnc_GetPriceMatched(const CLzmaProb *probs, UInt32 symbol, UInt32 matchByte, UInt32 *ProbPrices)
++{
++ UInt32 price = 0;
++ UInt32 offs = 0x100;
++ symbol |= 0x100;
++ do
++ {
++ matchByte <<= 1;
++ price += GET_PRICEa(probs[offs + (matchByte & offs) + (symbol >> 8)], (symbol >> 7) & 1);
++ symbol <<= 1;
++ offs &= ~(matchByte ^ symbol);
++ }
++ while (symbol < 0x10000);
++ return price;
++}
++
++
++static void RcTree_Encode(CRangeEnc *rc, CLzmaProb *probs, int numBitLevels, UInt32 symbol)
++{
++ UInt32 m = 1;
++ int i;
++ for (i = numBitLevels; i != 0;)
++ {
++ UInt32 bit;
++ i--;
++ bit = (symbol >> i) & 1;
++ RangeEnc_EncodeBit(rc, probs + m, bit);
++ m = (m << 1) | bit;
++ }
++}
++
++static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaProb *probs, int numBitLevels, UInt32 symbol)
++{
++ UInt32 m = 1;
++ int i;
++ for (i = 0; i < numBitLevels; i++)
++ {
++ UInt32 bit = symbol & 1;
++ RangeEnc_EncodeBit(rc, probs + m, bit);
++ m = (m << 1) | bit;
++ symbol >>= 1;
++ }
++}
++
++static UInt32 RcTree_GetPrice(const CLzmaProb *probs, int numBitLevels, UInt32 symbol, UInt32 *ProbPrices)
++{
++ UInt32 price = 0;
++ symbol |= (1 << numBitLevels);
++ while (symbol != 1)
++ {
++ price += GET_PRICEa(probs[symbol >> 1], symbol & 1);
++ symbol >>= 1;
++ }
++ return price;
++}
++
++static UInt32 RcTree_ReverseGetPrice(const CLzmaProb *probs, int numBitLevels, UInt32 symbol, UInt32 *ProbPrices)
++{
++ UInt32 price = 0;
++ UInt32 m = 1;
++ int i;
++ for (i = numBitLevels; i != 0; i--)
++ {
++ UInt32 bit = symbol & 1;
++ symbol >>= 1;
++ price += GET_PRICEa(probs[m], bit);
++ m = (m << 1) | bit;
++ }
++ return price;
++}
++
++
++static void LenEnc_Init(CLenEnc *p)
++{
++ unsigned i;
++ p->choice = p->choice2 = kProbInitValue;
++ for (i = 0; i < (LZMA_NUM_PB_STATES_MAX << kLenNumLowBits); i++)
++ p->low[i] = kProbInitValue;
++ for (i = 0; i < (LZMA_NUM_PB_STATES_MAX << kLenNumMidBits); i++)
++ p->mid[i] = kProbInitValue;
++ for (i = 0; i < kLenNumHighSymbols; i++)
++ p->high[i] = kProbInitValue;
++}
++
++static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, UInt32 symbol, UInt32 posState)
++{
++ if (symbol < kLenNumLowSymbols)
++ {
++ RangeEnc_EncodeBit(rc, &p->choice, 0);
++ RcTree_Encode(rc, p->low + (posState << kLenNumLowBits), kLenNumLowBits, symbol);
++ }
++ else
++ {
++ RangeEnc_EncodeBit(rc, &p->choice, 1);
++ if (symbol < kLenNumLowSymbols + kLenNumMidSymbols)
++ {
++ RangeEnc_EncodeBit(rc, &p->choice2, 0);
++ RcTree_Encode(rc, p->mid + (posState << kLenNumMidBits), kLenNumMidBits, symbol - kLenNumLowSymbols);
++ }
++ else
++ {
++ RangeEnc_EncodeBit(rc, &p->choice2, 1);
++ RcTree_Encode(rc, p->high, kLenNumHighBits, symbol - kLenNumLowSymbols - kLenNumMidSymbols);
++ }
++ }
++}
++
++static void LenEnc_SetPrices(CLenEnc *p, UInt32 posState, UInt32 numSymbols, UInt32 *prices, UInt32 *ProbPrices)
++{
++ UInt32 a0 = GET_PRICE_0a(p->choice);
++ UInt32 a1 = GET_PRICE_1a(p->choice);
++ UInt32 b0 = a1 + GET_PRICE_0a(p->choice2);
++ UInt32 b1 = a1 + GET_PRICE_1a(p->choice2);
++ UInt32 i = 0;
++ for (i = 0; i < kLenNumLowSymbols; i++)
++ {
++ if (i >= numSymbols)
++ return;
++ prices[i] = a0 + RcTree_GetPrice(p->low + (posState << kLenNumLowBits), kLenNumLowBits, i, ProbPrices);
++ }
++ for (; i < kLenNumLowSymbols + kLenNumMidSymbols; i++)
++ {
++ if (i >= numSymbols)
++ return;
++ prices[i] = b0 + RcTree_GetPrice(p->mid + (posState << kLenNumMidBits), kLenNumMidBits, i - kLenNumLowSymbols, ProbPrices);
++ }
++ for (; i < numSymbols; i++)
++ prices[i] = b1 + RcTree_GetPrice(p->high, kLenNumHighBits, i - kLenNumLowSymbols - kLenNumMidSymbols, ProbPrices);
++}
++
++static void MY_FAST_CALL LenPriceEnc_UpdateTable(CLenPriceEnc *p, UInt32 posState, UInt32 *ProbPrices)
++{
++ LenEnc_SetPrices(&p->p, posState, p->tableSize, p->prices[posState], ProbPrices);
++ p->counters[posState] = p->tableSize;
++}
++
++static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, UInt32 numPosStates, UInt32 *ProbPrices)
++{
++ UInt32 posState;
++ for (posState = 0; posState < numPosStates; posState++)
++ LenPriceEnc_UpdateTable(p, posState, ProbPrices);
++}
++
++static void LenEnc_Encode2(CLenPriceEnc *p, CRangeEnc *rc, UInt32 symbol, UInt32 posState, Bool updatePrice, UInt32 *ProbPrices)
++{
++ LenEnc_Encode(&p->p, rc, symbol, posState);
++ if (updatePrice)
++ if (--p->counters[posState] == 0)
++ LenPriceEnc_UpdateTable(p, posState, ProbPrices);
++}
++
++
++
++
++static void MovePos(CLzmaEnc *p, UInt32 num)
++{
++ #ifdef SHOW_STAT
++ ttt += num;
++ printf("\n MovePos %d", num);
++ #endif
++ if (num != 0)
++ {
++ p->additionalOffset += num;
++ p->matchFinder.Skip(p->matchFinderObj, num);
++ }
++}
++
++static UInt32 ReadMatchDistances(CLzmaEnc *p, UInt32 *numDistancePairsRes)
++{
++ UInt32 lenRes = 0, numPairs;
++ p->numAvail = p->matchFinder.GetNumAvailableBytes(p->matchFinderObj);
++ numPairs = p->matchFinder.GetMatches(p->matchFinderObj, p->matches);
++ #ifdef SHOW_STAT
++ printf("\n i = %d numPairs = %d ", ttt, numPairs / 2);
++ ttt++;
++ {
++ UInt32 i;
++ for (i = 0; i < numPairs; i += 2)
++ printf("%2d %6d | ", p->matches[i], p->matches[i + 1]);
++ }
++ #endif
++ if (numPairs > 0)
++ {
++ lenRes = p->matches[numPairs - 2];
++ if (lenRes == p->numFastBytes)
++ {
++ const Byte *pby = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
++ UInt32 distance = p->matches[numPairs - 1] + 1;
++ UInt32 numAvail = p->numAvail;
++ if (numAvail > LZMA_MATCH_LEN_MAX)
++ numAvail = LZMA_MATCH_LEN_MAX;
++ {
++ const Byte *pby2 = pby - distance;
++ for (; lenRes < numAvail && pby[lenRes] == pby2[lenRes]; lenRes++);
++ }
++ }
++ }
++ p->additionalOffset++;
++ *numDistancePairsRes = numPairs;
++ return lenRes;
++}
++
++
++#define MakeAsChar(p) (p)->backPrev = (UInt32)(-1); (p)->prev1IsChar = False;
++#define MakeAsShortRep(p) (p)->backPrev = 0; (p)->prev1IsChar = False;
++#define IsShortRep(p) ((p)->backPrev == 0)
++
++static UInt32 GetRepLen1Price(CLzmaEnc *p, UInt32 state, UInt32 posState)
++{
++ return
++ GET_PRICE_0(p->isRepG0[state]) +
++ GET_PRICE_0(p->isRep0Long[state][posState]);
++}
++
++static UInt32 GetPureRepPrice(CLzmaEnc *p, UInt32 repIndex, UInt32 state, UInt32 posState)
++{
++ UInt32 price;
++ if (repIndex == 0)
++ {
++ price = GET_PRICE_0(p->isRepG0[state]);
++ price += GET_PRICE_1(p->isRep0Long[state][posState]);
++ }
++ else
++ {
++ price = GET_PRICE_1(p->isRepG0[state]);
++ if (repIndex == 1)
++ price += GET_PRICE_0(p->isRepG1[state]);
++ else
++ {
++ price += GET_PRICE_1(p->isRepG1[state]);
++ price += GET_PRICE(p->isRepG2[state], repIndex - 2);
++ }
++ }
++ return price;
++}
++
++static UInt32 GetRepPrice(CLzmaEnc *p, UInt32 repIndex, UInt32 len, UInt32 state, UInt32 posState)
++{
++ return p->repLenEnc.prices[posState][len - LZMA_MATCH_LEN_MIN] +
++ GetPureRepPrice(p, repIndex, state, posState);
++}
++
++static UInt32 Backward(CLzmaEnc *p, UInt32 *backRes, UInt32 cur)
++{
++ UInt32 posMem = p->opt[cur].posPrev;
++ UInt32 backMem = p->opt[cur].backPrev;
++ p->optimumEndIndex = cur;
++ do
++ {
++ if (p->opt[cur].prev1IsChar)
++ {
++ MakeAsChar(&p->opt[posMem])
++ p->opt[posMem].posPrev = posMem - 1;
++ if (p->opt[cur].prev2)
++ {
++ p->opt[posMem - 1].prev1IsChar = False;
++ p->opt[posMem - 1].posPrev = p->opt[cur].posPrev2;
++ p->opt[posMem - 1].backPrev = p->opt[cur].backPrev2;
++ }
++ }
++ {
++ UInt32 posPrev = posMem;
++ UInt32 backCur = backMem;
++
++ backMem = p->opt[posPrev].backPrev;
++ posMem = p->opt[posPrev].posPrev;
++
++ p->opt[posPrev].backPrev = backCur;
++ p->opt[posPrev].posPrev = cur;
++ cur = posPrev;
++ }
++ }
++ while (cur != 0);
++ *backRes = p->opt[0].backPrev;
++ p->optimumCurrentIndex = p->opt[0].posPrev;
++ return p->optimumCurrentIndex;
++}
++
++#define LIT_PROBS(pos, prevByte) (p->litProbs + ((((pos) & p->lpMask) << p->lc) + ((prevByte) >> (8 - p->lc))) * 0x300)
++
++static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position, UInt32 *backRes)
++{
++ UInt32 numAvail, mainLen, numPairs, repMaxIndex, i, posState, lenEnd, len, cur;
++ UInt32 matchPrice, repMatchPrice, normalMatchPrice;
++ UInt32 reps[LZMA_NUM_REPS], repLens[LZMA_NUM_REPS];
++ UInt32 *matches;
++ const Byte *data;
++ Byte curByte, matchByte;
++ if (p->optimumEndIndex != p->optimumCurrentIndex)
++ {
++ const COptimal *opt = &p->opt[p->optimumCurrentIndex];
++ UInt32 lenRes = opt->posPrev - p->optimumCurrentIndex;
++ *backRes = opt->backPrev;
++ p->optimumCurrentIndex = opt->posPrev;
++ return lenRes;
++ }
++ p->optimumCurrentIndex = p->optimumEndIndex = 0;
++
++ if (p->additionalOffset == 0)
++ mainLen = ReadMatchDistances(p, &numPairs);
++ else
++ {
++ mainLen = p->longestMatchLength;
++ numPairs = p->numPairs;
++ }
++
++ numAvail = p->numAvail;
++ if (numAvail < 2)
++ {
++ *backRes = (UInt32)(-1);
++ return 1;
++ }
++ if (numAvail > LZMA_MATCH_LEN_MAX)
++ numAvail = LZMA_MATCH_LEN_MAX;
++
++ data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
++ repMaxIndex = 0;
++ for (i = 0; i < LZMA_NUM_REPS; i++)
++ {
++ UInt32 lenTest;
++ const Byte *data2;
++ reps[i] = p->reps[i];
++ data2 = data - (reps[i] + 1);
++ if (data[0] != data2[0] || data[1] != data2[1])
++ {
++ repLens[i] = 0;
++ continue;
++ }
++ for (lenTest = 2; lenTest < numAvail && data[lenTest] == data2[lenTest]; lenTest++);
++ repLens[i] = lenTest;
++ if (lenTest > repLens[repMaxIndex])
++ repMaxIndex = i;
++ }
++ if (repLens[repMaxIndex] >= p->numFastBytes)
++ {
++ UInt32 lenRes;
++ *backRes = repMaxIndex;
++ lenRes = repLens[repMaxIndex];
++ MovePos(p, lenRes - 1);
++ return lenRes;
++ }
++
++ matches = p->matches;
++ if (mainLen >= p->numFastBytes)
++ {
++ *backRes = matches[numPairs - 1] + LZMA_NUM_REPS;
++ MovePos(p, mainLen - 1);
++ return mainLen;
++ }
++ curByte = *data;
++ matchByte = *(data - (reps[0] + 1));
++
++ if (mainLen < 2 && curByte != matchByte && repLens[repMaxIndex] < 2)
++ {
++ *backRes = (UInt32)-1;
++ return 1;
++ }
++
++ p->opt[0].state = (CState)p->state;
++
++ posState = (position & p->pbMask);
++
++ {
++ const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
++ p->opt[1].price = GET_PRICE_0(p->isMatch[p->state][posState]) +
++ (!IsCharState(p->state) ?
++ LitEnc_GetPriceMatched(probs, curByte, matchByte, p->ProbPrices) :
++ LitEnc_GetPrice(probs, curByte, p->ProbPrices));
++ }
++
++ MakeAsChar(&p->opt[1]);
++
++ matchPrice = GET_PRICE_1(p->isMatch[p->state][posState]);
++ repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[p->state]);
++
++ if (matchByte == curByte)
++ {
++ UInt32 shortRepPrice = repMatchPrice + GetRepLen1Price(p, p->state, posState);
++ if (shortRepPrice < p->opt[1].price)
++ {
++ p->opt[1].price = shortRepPrice;
++ MakeAsShortRep(&p->opt[1]);
++ }
++ }
++ lenEnd = ((mainLen >= repLens[repMaxIndex]) ? mainLen : repLens[repMaxIndex]);
++
++ if (lenEnd < 2)
++ {
++ *backRes = p->opt[1].backPrev;
++ return 1;
++ }
++
++ p->opt[1].posPrev = 0;
++ for (i = 0; i < LZMA_NUM_REPS; i++)
++ p->opt[0].backs[i] = reps[i];
++
++ len = lenEnd;
++ do
++ p->opt[len--].price = kInfinityPrice;
++ while (len >= 2);
++
++ for (i = 0; i < LZMA_NUM_REPS; i++)
++ {
++ UInt32 repLen = repLens[i];
++ UInt32 price;
++ if (repLen < 2)
++ continue;
++ price = repMatchPrice + GetPureRepPrice(p, i, p->state, posState);
++ do
++ {
++ UInt32 curAndLenPrice = price + p->repLenEnc.prices[posState][repLen - 2];
++ COptimal *opt = &p->opt[repLen];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = 0;
++ opt->backPrev = i;
++ opt->prev1IsChar = False;
++ }
++ }
++ while (--repLen >= 2);
++ }
++
++ normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[p->state]);
++
++ len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2);
++ if (len <= mainLen)
++ {
++ UInt32 offs = 0;
++ while (len > matches[offs])
++ offs += 2;
++ for (; ; len++)
++ {
++ COptimal *opt;
++ UInt32 distance = matches[offs + 1];
++
++ UInt32 curAndLenPrice = normalMatchPrice + p->lenEnc.prices[posState][len - LZMA_MATCH_LEN_MIN];
++ UInt32 lenToPosState = GetLenToPosState(len);
++ if (distance < kNumFullDistances)
++ curAndLenPrice += p->distancesPrices[lenToPosState][distance];
++ else
++ {
++ UInt32 slot;
++ GetPosSlot2(distance, slot);
++ curAndLenPrice += p->alignPrices[distance & kAlignMask] + p->posSlotPrices[lenToPosState][slot];
++ }
++ opt = &p->opt[len];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = 0;
++ opt->backPrev = distance + LZMA_NUM_REPS;
++ opt->prev1IsChar = False;
++ }
++ if (len == matches[offs])
++ {
++ offs += 2;
++ if (offs == numPairs)
++ break;
++ }
++ }
++ }
++
++ cur = 0;
++
++ #ifdef SHOW_STAT2
++ if (position >= 0)
++ {
++ unsigned i;
++ printf("\n pos = %4X", position);
++ for (i = cur; i <= lenEnd; i++)
++ printf("\nprice[%4X] = %d", position - cur + i, p->opt[i].price);
++ }
++ #endif
++
++ for (;;)
++ {
++ UInt32 numAvailFull, newLen, numPairs, posPrev, state, posState, startLen;
++ UInt32 curPrice, curAnd1Price, matchPrice, repMatchPrice;
++ Bool nextIsChar;
++ Byte curByte, matchByte;
++ const Byte *data;
++ COptimal *curOpt;
++ COptimal *nextOpt;
++
++ cur++;
++ if (cur == lenEnd)
++ return Backward(p, backRes, cur);
++
++ newLen = ReadMatchDistances(p, &numPairs);
++ if (newLen >= p->numFastBytes)
++ {
++ p->numPairs = numPairs;
++ p->longestMatchLength = newLen;
++ return Backward(p, backRes, cur);
++ }
++ position++;
++ curOpt = &p->opt[cur];
++ posPrev = curOpt->posPrev;
++ if (curOpt->prev1IsChar)
++ {
++ posPrev--;
++ if (curOpt->prev2)
++ {
++ state = p->opt[curOpt->posPrev2].state;
++ if (curOpt->backPrev2 < LZMA_NUM_REPS)
++ state = kRepNextStates[state];
++ else
++ state = kMatchNextStates[state];
++ }
++ else
++ state = p->opt[posPrev].state;
++ state = kLiteralNextStates[state];
++ }
++ else
++ state = p->opt[posPrev].state;
++ if (posPrev == cur - 1)
++ {
++ if (IsShortRep(curOpt))
++ state = kShortRepNextStates[state];
++ else
++ state = kLiteralNextStates[state];
++ }
++ else
++ {
++ UInt32 pos;
++ const COptimal *prevOpt;
++ if (curOpt->prev1IsChar && curOpt->prev2)
++ {
++ posPrev = curOpt->posPrev2;
++ pos = curOpt->backPrev2;
++ state = kRepNextStates[state];
++ }
++ else
++ {
++ pos = curOpt->backPrev;
++ if (pos < LZMA_NUM_REPS)
++ state = kRepNextStates[state];
++ else
++ state = kMatchNextStates[state];
++ }
++ prevOpt = &p->opt[posPrev];
++ if (pos < LZMA_NUM_REPS)
++ {
++ UInt32 i;
++ reps[0] = prevOpt->backs[pos];
++ for (i = 1; i <= pos; i++)
++ reps[i] = prevOpt->backs[i - 1];
++ for (; i < LZMA_NUM_REPS; i++)
++ reps[i] = prevOpt->backs[i];
++ }
++ else
++ {
++ UInt32 i;
++ reps[0] = (pos - LZMA_NUM_REPS);
++ for (i = 1; i < LZMA_NUM_REPS; i++)
++ reps[i] = prevOpt->backs[i - 1];
++ }
++ }
++ curOpt->state = (CState)state;
++
++ curOpt->backs[0] = reps[0];
++ curOpt->backs[1] = reps[1];
++ curOpt->backs[2] = reps[2];
++ curOpt->backs[3] = reps[3];
++
++ curPrice = curOpt->price;
++ nextIsChar = False;
++ data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
++ curByte = *data;
++ matchByte = *(data - (reps[0] + 1));
++
++ posState = (position & p->pbMask);
++
++ curAnd1Price = curPrice + GET_PRICE_0(p->isMatch[state][posState]);
++ {
++ const CLzmaProb *probs = LIT_PROBS(position, *(data - 1));
++ curAnd1Price +=
++ (!IsCharState(state) ?
++ LitEnc_GetPriceMatched(probs, curByte, matchByte, p->ProbPrices) :
++ LitEnc_GetPrice(probs, curByte, p->ProbPrices));
++ }
++
++ nextOpt = &p->opt[cur + 1];
++
++ if (curAnd1Price < nextOpt->price)
++ {
++ nextOpt->price = curAnd1Price;
++ nextOpt->posPrev = cur;
++ MakeAsChar(nextOpt);
++ nextIsChar = True;
++ }
++
++ matchPrice = curPrice + GET_PRICE_1(p->isMatch[state][posState]);
++ repMatchPrice = matchPrice + GET_PRICE_1(p->isRep[state]);
++
++ if (matchByte == curByte && !(nextOpt->posPrev < cur && nextOpt->backPrev == 0))
++ {
++ UInt32 shortRepPrice = repMatchPrice + GetRepLen1Price(p, state, posState);
++ if (shortRepPrice <= nextOpt->price)
++ {
++ nextOpt->price = shortRepPrice;
++ nextOpt->posPrev = cur;
++ MakeAsShortRep(nextOpt);
++ nextIsChar = True;
++ }
++ }
++ numAvailFull = p->numAvail;
++ {
++ UInt32 temp = kNumOpts - 1 - cur;
++ if (temp < numAvailFull)
++ numAvailFull = temp;
++ }
++
++ if (numAvailFull < 2)
++ continue;
++ numAvail = (numAvailFull <= p->numFastBytes ? numAvailFull : p->numFastBytes);
++
++ if (!nextIsChar && matchByte != curByte) /* speed optimization */
++ {
++ /* try Literal + rep0 */
++ UInt32 temp;
++ UInt32 lenTest2;
++ const Byte *data2 = data - (reps[0] + 1);
++ UInt32 limit = p->numFastBytes + 1;
++ if (limit > numAvailFull)
++ limit = numAvailFull;
++
++ for (temp = 1; temp < limit && data[temp] == data2[temp]; temp++);
++ lenTest2 = temp - 1;
++ if (lenTest2 >= 2)
++ {
++ UInt32 state2 = kLiteralNextStates[state];
++ UInt32 posStateNext = (position + 1) & p->pbMask;
++ UInt32 nextRepMatchPrice = curAnd1Price +
++ GET_PRICE_1(p->isMatch[state2][posStateNext]) +
++ GET_PRICE_1(p->isRep[state2]);
++ /* for (; lenTest2 >= 2; lenTest2--) */
++ {
++ UInt32 curAndLenPrice;
++ COptimal *opt;
++ UInt32 offset = cur + 1 + lenTest2;
++ while (lenEnd < offset)
++ p->opt[++lenEnd].price = kInfinityPrice;
++ curAndLenPrice = nextRepMatchPrice + GetRepPrice(p, 0, lenTest2, state2, posStateNext);
++ opt = &p->opt[offset];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = cur + 1;
++ opt->backPrev = 0;
++ opt->prev1IsChar = True;
++ opt->prev2 = False;
++ }
++ }
++ }
++ }
++
++ startLen = 2; /* speed optimization */
++ {
++ UInt32 repIndex;
++ for (repIndex = 0; repIndex < LZMA_NUM_REPS; repIndex++)
++ {
++ UInt32 lenTest;
++ UInt32 lenTestTemp;
++ UInt32 price;
++ const Byte *data2 = data - (reps[repIndex] + 1);
++ if (data[0] != data2[0] || data[1] != data2[1])
++ continue;
++ for (lenTest = 2; lenTest < numAvail && data[lenTest] == data2[lenTest]; lenTest++);
++ while (lenEnd < cur + lenTest)
++ p->opt[++lenEnd].price = kInfinityPrice;
++ lenTestTemp = lenTest;
++ price = repMatchPrice + GetPureRepPrice(p, repIndex, state, posState);
++ do
++ {
++ UInt32 curAndLenPrice = price + p->repLenEnc.prices[posState][lenTest - 2];
++ COptimal *opt = &p->opt[cur + lenTest];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = cur;
++ opt->backPrev = repIndex;
++ opt->prev1IsChar = False;
++ }
++ }
++ while (--lenTest >= 2);
++ lenTest = lenTestTemp;
++
++ if (repIndex == 0)
++ startLen = lenTest + 1;
++
++ /* if (_maxMode) */
++ {
++ UInt32 lenTest2 = lenTest + 1;
++ UInt32 limit = lenTest2 + p->numFastBytes;
++ UInt32 nextRepMatchPrice;
++ if (limit > numAvailFull)
++ limit = numAvailFull;
++ for (; lenTest2 < limit && data[lenTest2] == data2[lenTest2]; lenTest2++);
++ lenTest2 -= lenTest + 1;
++ if (lenTest2 >= 2)
++ {
++ UInt32 state2 = kRepNextStates[state];
++ UInt32 posStateNext = (position + lenTest) & p->pbMask;
++ UInt32 curAndLenCharPrice =
++ price + p->repLenEnc.prices[posState][lenTest - 2] +
++ GET_PRICE_0(p->isMatch[state2][posStateNext]) +
++ LitEnc_GetPriceMatched(LIT_PROBS(position + lenTest, data[lenTest - 1]),
++ data[lenTest], data2[lenTest], p->ProbPrices);
++ state2 = kLiteralNextStates[state2];
++ posStateNext = (position + lenTest + 1) & p->pbMask;
++ nextRepMatchPrice = curAndLenCharPrice +
++ GET_PRICE_1(p->isMatch[state2][posStateNext]) +
++ GET_PRICE_1(p->isRep[state2]);
++
++ /* for (; lenTest2 >= 2; lenTest2--) */
++ {
++ UInt32 curAndLenPrice;
++ COptimal *opt;
++ UInt32 offset = cur + lenTest + 1 + lenTest2;
++ while (lenEnd < offset)
++ p->opt[++lenEnd].price = kInfinityPrice;
++ curAndLenPrice = nextRepMatchPrice + GetRepPrice(p, 0, lenTest2, state2, posStateNext);
++ opt = &p->opt[offset];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = cur + lenTest + 1;
++ opt->backPrev = 0;
++ opt->prev1IsChar = True;
++ opt->prev2 = True;
++ opt->posPrev2 = cur;
++ opt->backPrev2 = repIndex;
++ }
++ }
++ }
++ }
++ }
++ }
++ /* for (UInt32 lenTest = 2; lenTest <= newLen; lenTest++) */
++ if (newLen > numAvail)
++ {
++ newLen = numAvail;
++ for (numPairs = 0; newLen > matches[numPairs]; numPairs += 2);
++ matches[numPairs] = newLen;
++ numPairs += 2;
++ }
++ if (newLen >= startLen)
++ {
++ UInt32 normalMatchPrice = matchPrice + GET_PRICE_0(p->isRep[state]);
++ UInt32 offs, curBack, posSlot;
++ UInt32 lenTest;
++ while (lenEnd < cur + newLen)
++ p->opt[++lenEnd].price = kInfinityPrice;
++
++ offs = 0;
++ while (startLen > matches[offs])
++ offs += 2;
++ curBack = matches[offs + 1];
++ GetPosSlot2(curBack, posSlot);
++ for (lenTest = /*2*/ startLen; ; lenTest++)
++ {
++ UInt32 curAndLenPrice = normalMatchPrice + p->lenEnc.prices[posState][lenTest - LZMA_MATCH_LEN_MIN];
++ UInt32 lenToPosState = GetLenToPosState(lenTest);
++ COptimal *opt;
++ if (curBack < kNumFullDistances)
++ curAndLenPrice += p->distancesPrices[lenToPosState][curBack];
++ else
++ curAndLenPrice += p->posSlotPrices[lenToPosState][posSlot] + p->alignPrices[curBack & kAlignMask];
++
++ opt = &p->opt[cur + lenTest];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = cur;
++ opt->backPrev = curBack + LZMA_NUM_REPS;
++ opt->prev1IsChar = False;
++ }
++
++ if (/*_maxMode && */lenTest == matches[offs])
++ {
++ /* Try Match + Literal + Rep0 */
++ const Byte *data2 = data - (curBack + 1);
++ UInt32 lenTest2 = lenTest + 1;
++ UInt32 limit = lenTest2 + p->numFastBytes;
++ UInt32 nextRepMatchPrice;
++ if (limit > numAvailFull)
++ limit = numAvailFull;
++ for (; lenTest2 < limit && data[lenTest2] == data2[lenTest2]; lenTest2++);
++ lenTest2 -= lenTest + 1;
++ if (lenTest2 >= 2)
++ {
++ UInt32 state2 = kMatchNextStates[state];
++ UInt32 posStateNext = (position + lenTest) & p->pbMask;
++ UInt32 curAndLenCharPrice = curAndLenPrice +
++ GET_PRICE_0(p->isMatch[state2][posStateNext]) +
++ LitEnc_GetPriceMatched(LIT_PROBS(position + lenTest, data[lenTest - 1]),
++ data[lenTest], data2[lenTest], p->ProbPrices);
++ state2 = kLiteralNextStates[state2];
++ posStateNext = (posStateNext + 1) & p->pbMask;
++ nextRepMatchPrice = curAndLenCharPrice +
++ GET_PRICE_1(p->isMatch[state2][posStateNext]) +
++ GET_PRICE_1(p->isRep[state2]);
++
++ /* for (; lenTest2 >= 2; lenTest2--) */
++ {
++ UInt32 offset = cur + lenTest + 1 + lenTest2;
++ UInt32 curAndLenPrice;
++ COptimal *opt;
++ while (lenEnd < offset)
++ p->opt[++lenEnd].price = kInfinityPrice;
++ curAndLenPrice = nextRepMatchPrice + GetRepPrice(p, 0, lenTest2, state2, posStateNext);
++ opt = &p->opt[offset];
++ if (curAndLenPrice < opt->price)
++ {
++ opt->price = curAndLenPrice;
++ opt->posPrev = cur + lenTest + 1;
++ opt->backPrev = 0;
++ opt->prev1IsChar = True;
++ opt->prev2 = True;
++ opt->posPrev2 = cur;
++ opt->backPrev2 = curBack + LZMA_NUM_REPS;
++ }
++ }
++ }
++ offs += 2;
++ if (offs == numPairs)
++ break;
++ curBack = matches[offs + 1];
++ if (curBack >= kNumFullDistances)
++ GetPosSlot2(curBack, posSlot);
++ }
++ }
++ }
++ }
++}
++
++#define ChangePair(smallDist, bigDist) (((bigDist) >> 7) > (smallDist))
++
++static UInt32 GetOptimumFast(CLzmaEnc *p, UInt32 *backRes)
++{
++ UInt32 numAvail, mainLen, mainDist, numPairs, repIndex, repLen, i;
++ const Byte *data;
++ const UInt32 *matches;
++
++ if (p->additionalOffset == 0)
++ mainLen = ReadMatchDistances(p, &numPairs);
++ else
++ {
++ mainLen = p->longestMatchLength;
++ numPairs = p->numPairs;
++ }
++
++ numAvail = p->numAvail;
++ *backRes = (UInt32)-1;
++ if (numAvail < 2)
++ return 1;
++ if (numAvail > LZMA_MATCH_LEN_MAX)
++ numAvail = LZMA_MATCH_LEN_MAX;
++ data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
++
++ repLen = repIndex = 0;
++ for (i = 0; i < LZMA_NUM_REPS; i++)
++ {
++ UInt32 len;
++ const Byte *data2 = data - (p->reps[i] + 1);
++ if (data[0] != data2[0] || data[1] != data2[1])
++ continue;
++ for (len = 2; len < numAvail && data[len] == data2[len]; len++);
++ if (len >= p->numFastBytes)
++ {
++ *backRes = i;
++ MovePos(p, len - 1);
++ return len;
++ }
++ if (len > repLen)
++ {
++ repIndex = i;
++ repLen = len;
++ }
++ }
++
++ matches = p->matches;
++ if (mainLen >= p->numFastBytes)
++ {
++ *backRes = matches[numPairs - 1] + LZMA_NUM_REPS;
++ MovePos(p, mainLen - 1);
++ return mainLen;
++ }
++
++ mainDist = 0; /* for GCC */
++ if (mainLen >= 2)
++ {
++ mainDist = matches[numPairs - 1];
++ while (numPairs > 2 && mainLen == matches[numPairs - 4] + 1)
++ {
++ if (!ChangePair(matches[numPairs - 3], mainDist))
++ break;
++ numPairs -= 2;
++ mainLen = matches[numPairs - 2];
++ mainDist = matches[numPairs - 1];
++ }
++ if (mainLen == 2 && mainDist >= 0x80)
++ mainLen = 1;
++ }
++
++ if (repLen >= 2 && (
++ (repLen + 1 >= mainLen) ||
++ (repLen + 2 >= mainLen && mainDist >= (1 << 9)) ||
++ (repLen + 3 >= mainLen && mainDist >= (1 << 15))))
++ {
++ *backRes = repIndex;
++ MovePos(p, repLen - 1);
++ return repLen;
++ }
++
++ if (mainLen < 2 || numAvail <= 2)
++ return 1;
++
++ p->longestMatchLength = ReadMatchDistances(p, &p->numPairs);
++ if (p->longestMatchLength >= 2)
++ {
++ UInt32 newDistance = matches[p->numPairs - 1];
++ if ((p->longestMatchLength >= mainLen && newDistance < mainDist) ||
++ (p->longestMatchLength == mainLen + 1 && !ChangePair(mainDist, newDistance)) ||
++ (p->longestMatchLength > mainLen + 1) ||
++ (p->longestMatchLength + 1 >= mainLen && mainLen >= 3 && ChangePair(newDistance, mainDist)))
++ return 1;
++ }
++
++ data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - 1;
++ for (i = 0; i < LZMA_NUM_REPS; i++)
++ {
++ UInt32 len, limit;
++ const Byte *data2 = data - (p->reps[i] + 1);
++ if (data[0] != data2[0] || data[1] != data2[1])
++ continue;
++ limit = mainLen - 1;
++ for (len = 2; len < limit && data[len] == data2[len]; len++);
++ if (len >= limit)
++ return 1;
++ }
++ *backRes = mainDist + LZMA_NUM_REPS;
++ MovePos(p, mainLen - 2);
++ return mainLen;
++}
++
++static void WriteEndMarker(CLzmaEnc *p, UInt32 posState)
++{
++ UInt32 len;
++ RangeEnc_EncodeBit(&p->rc, &p->isMatch[p->state][posState], 1);
++ RangeEnc_EncodeBit(&p->rc, &p->isRep[p->state], 0);
++ p->state = kMatchNextStates[p->state];
++ len = LZMA_MATCH_LEN_MIN;
++ LenEnc_Encode2(&p->lenEnc, &p->rc, len - LZMA_MATCH_LEN_MIN, posState, !p->fastMode, p->ProbPrices);
++ RcTree_Encode(&p->rc, p->posSlotEncoder[GetLenToPosState(len)], kNumPosSlotBits, (1 << kNumPosSlotBits) - 1);
++ RangeEnc_EncodeDirectBits(&p->rc, (((UInt32)1 << 30) - 1) >> kNumAlignBits, 30 - kNumAlignBits);
++ RcTree_ReverseEncode(&p->rc, p->posAlignEncoder, kNumAlignBits, kAlignMask);
++}
++
++static SRes CheckErrors(CLzmaEnc *p)
++{
++ if (p->result != SZ_OK)
++ return p->result;
++ if (p->rc.res != SZ_OK)
++ p->result = SZ_ERROR_WRITE;
++ if (p->matchFinderBase.result != SZ_OK)
++ p->result = SZ_ERROR_READ;
++ if (p->result != SZ_OK)
++ p->finished = True;
++ return p->result;
++}
++
++static SRes Flush(CLzmaEnc *p, UInt32 nowPos)
++{
++ /* ReleaseMFStream(); */
++ p->finished = True;
++ if (p->writeEndMark)
++ WriteEndMarker(p, nowPos & p->pbMask);
++ RangeEnc_FlushData(&p->rc);
++ RangeEnc_FlushStream(&p->rc);
++ return CheckErrors(p);
++}
++
++static void FillAlignPrices(CLzmaEnc *p)
++{
++ UInt32 i;
++ for (i = 0; i < kAlignTableSize; i++)
++ p->alignPrices[i] = RcTree_ReverseGetPrice(p->posAlignEncoder, kNumAlignBits, i, p->ProbPrices);
++ p->alignPriceCount = 0;
++}
++
++static void FillDistancesPrices(CLzmaEnc *p)
++{
++ UInt32 tempPrices[kNumFullDistances];
++ UInt32 i, lenToPosState;
++ for (i = kStartPosModelIndex; i < kNumFullDistances; i++)
++ {
++ UInt32 posSlot = GetPosSlot1(i);
++ UInt32 footerBits = ((posSlot >> 1) - 1);
++ UInt32 base = ((2 | (posSlot & 1)) << footerBits);
++ tempPrices[i] = RcTree_ReverseGetPrice(p->posEncoders + base - posSlot - 1, footerBits, i - base, p->ProbPrices);
++ }
++
++ for (lenToPosState = 0; lenToPosState < kNumLenToPosStates; lenToPosState++)
++ {
++ UInt32 posSlot;
++ const CLzmaProb *encoder = p->posSlotEncoder[lenToPosState];
++ UInt32 *posSlotPrices = p->posSlotPrices[lenToPosState];
++ for (posSlot = 0; posSlot < p->distTableSize; posSlot++)
++ posSlotPrices[posSlot] = RcTree_GetPrice(encoder, kNumPosSlotBits, posSlot, p->ProbPrices);
++ for (posSlot = kEndPosModelIndex; posSlot < p->distTableSize; posSlot++)
++ posSlotPrices[posSlot] += ((((posSlot >> 1) - 1) - kNumAlignBits) << kNumBitPriceShiftBits);
++
++ {
++ UInt32 *distancesPrices = p->distancesPrices[lenToPosState];
++ UInt32 i;
++ for (i = 0; i < kStartPosModelIndex; i++)
++ distancesPrices[i] = posSlotPrices[i];
++ for (; i < kNumFullDistances; i++)
++ distancesPrices[i] = posSlotPrices[GetPosSlot1(i)] + tempPrices[i];
++ }
++ }
++ p->matchPriceCount = 0;
++}
++
++void LzmaEnc_Construct(CLzmaEnc *p)
++{
++ RangeEnc_Construct(&p->rc);
++ MatchFinder_Construct(&p->matchFinderBase);
++ #ifndef _7ZIP_ST
++ MatchFinderMt_Construct(&p->matchFinderMt);
++ p->matchFinderMt.MatchFinder = &p->matchFinderBase;
++ #endif
++
++ {
++ CLzmaEncProps props;
++ LzmaEncProps_Init(&props);
++ LzmaEnc_SetProps(p, &props);
++ }
++
++ #ifndef LZMA_LOG_BSR
++ LzmaEnc_FastPosInit(p->g_FastPos);
++ #endif
++
++ LzmaEnc_InitPriceTables(p->ProbPrices);
++ p->litProbs = 0;
++ p->saveState.litProbs = 0;
++}
++
++CLzmaEncHandle LzmaEnc_Create(ISzAlloc *alloc)
++{
++ void *p;
++ p = alloc->Alloc(alloc, sizeof(CLzmaEnc));
++ if (p != 0)
++ LzmaEnc_Construct((CLzmaEnc *)p);
++ return p;
++}
++
++void LzmaEnc_FreeLits(CLzmaEnc *p, ISzAlloc *alloc)
++{
++ alloc->Free(alloc, p->litProbs);
++ alloc->Free(alloc, p->saveState.litProbs);
++ p->litProbs = 0;
++ p->saveState.litProbs = 0;
++}
++
++void LzmaEnc_Destruct(CLzmaEnc *p, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ #ifndef _7ZIP_ST
++ MatchFinderMt_Destruct(&p->matchFinderMt, allocBig);
++ #endif
++ MatchFinder_Free(&p->matchFinderBase, allocBig);
++ LzmaEnc_FreeLits(p, alloc);
++ RangeEnc_Free(&p->rc, alloc);
++}
++
++void LzmaEnc_Destroy(CLzmaEncHandle p, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ LzmaEnc_Destruct((CLzmaEnc *)p, alloc, allocBig);
++ alloc->Free(alloc, p);
++}
++
++static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, Bool useLimits, UInt32 maxPackSize, UInt32 maxUnpackSize)
++{
++ UInt32 nowPos32, startPos32;
++ if (p->needInit)
++ {
++ p->matchFinder.Init(p->matchFinderObj);
++ p->needInit = 0;
++ }
++
++ if (p->finished)
++ return p->result;
++ RINOK(CheckErrors(p));
++
++ nowPos32 = (UInt32)p->nowPos64;
++ startPos32 = nowPos32;
++
++ if (p->nowPos64 == 0)
++ {
++ UInt32 numPairs;
++ Byte curByte;
++ if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
++ return Flush(p, nowPos32);
++ ReadMatchDistances(p, &numPairs);
++ RangeEnc_EncodeBit(&p->rc, &p->isMatch[p->state][0], 0);
++ p->state = kLiteralNextStates[p->state];
++ curByte = p->matchFinder.GetIndexByte(p->matchFinderObj, 0 - p->additionalOffset);
++ LitEnc_Encode(&p->rc, p->litProbs, curByte);
++ p->additionalOffset--;
++ nowPos32++;
++ }
++
++ if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) != 0)
++ for (;;)
++ {
++ UInt32 pos, len, posState;
++
++ if (p->fastMode)
++ len = GetOptimumFast(p, &pos);
++ else
++ len = GetOptimum(p, nowPos32, &pos);
++
++ #ifdef SHOW_STAT2
++ printf("\n pos = %4X, len = %d pos = %d", nowPos32, len, pos);
++ #endif
++
++ posState = nowPos32 & p->pbMask;
++ if (len == 1 && pos == (UInt32)-1)
++ {
++ Byte curByte;
++ CLzmaProb *probs;
++ const Byte *data;
++
++ RangeEnc_EncodeBit(&p->rc, &p->isMatch[p->state][posState], 0);
++ data = p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - p->additionalOffset;
++ curByte = *data;
++ probs = LIT_PROBS(nowPos32, *(data - 1));
++ if (IsCharState(p->state))
++ LitEnc_Encode(&p->rc, probs, curByte);
++ else
++ LitEnc_EncodeMatched(&p->rc, probs, curByte, *(data - p->reps[0] - 1));
++ p->state = kLiteralNextStates[p->state];
++ }
++ else
++ {
++ RangeEnc_EncodeBit(&p->rc, &p->isMatch[p->state][posState], 1);
++ if (pos < LZMA_NUM_REPS)
++ {
++ RangeEnc_EncodeBit(&p->rc, &p->isRep[p->state], 1);
++ if (pos == 0)
++ {
++ RangeEnc_EncodeBit(&p->rc, &p->isRepG0[p->state], 0);
++ RangeEnc_EncodeBit(&p->rc, &p->isRep0Long[p->state][posState], ((len == 1) ? 0 : 1));
++ }
++ else
++ {
++ UInt32 distance = p->reps[pos];
++ RangeEnc_EncodeBit(&p->rc, &p->isRepG0[p->state], 1);
++ if (pos == 1)
++ RangeEnc_EncodeBit(&p->rc, &p->isRepG1[p->state], 0);
++ else
++ {
++ RangeEnc_EncodeBit(&p->rc, &p->isRepG1[p->state], 1);
++ RangeEnc_EncodeBit(&p->rc, &p->isRepG2[p->state], pos - 2);
++ if (pos == 3)
++ p->reps[3] = p->reps[2];
++ p->reps[2] = p->reps[1];
++ }
++ p->reps[1] = p->reps[0];
++ p->reps[0] = distance;
++ }
++ if (len == 1)
++ p->state = kShortRepNextStates[p->state];
++ else
++ {
++ LenEnc_Encode2(&p->repLenEnc, &p->rc, len - LZMA_MATCH_LEN_MIN, posState, !p->fastMode, p->ProbPrices);
++ p->state = kRepNextStates[p->state];
++ }
++ }
++ else
++ {
++ UInt32 posSlot;
++ RangeEnc_EncodeBit(&p->rc, &p->isRep[p->state], 0);
++ p->state = kMatchNextStates[p->state];
++ LenEnc_Encode2(&p->lenEnc, &p->rc, len - LZMA_MATCH_LEN_MIN, posState, !p->fastMode, p->ProbPrices);
++ pos -= LZMA_NUM_REPS;
++ GetPosSlot(pos, posSlot);
++ RcTree_Encode(&p->rc, p->posSlotEncoder[GetLenToPosState(len)], kNumPosSlotBits, posSlot);
++
++ if (posSlot >= kStartPosModelIndex)
++ {
++ UInt32 footerBits = ((posSlot >> 1) - 1);
++ UInt32 base = ((2 | (posSlot & 1)) << footerBits);
++ UInt32 posReduced = pos - base;
++
++ if (posSlot < kEndPosModelIndex)
++ RcTree_ReverseEncode(&p->rc, p->posEncoders + base - posSlot - 1, footerBits, posReduced);
++ else
++ {
++ RangeEnc_EncodeDirectBits(&p->rc, posReduced >> kNumAlignBits, footerBits - kNumAlignBits);
++ RcTree_ReverseEncode(&p->rc, p->posAlignEncoder, kNumAlignBits, posReduced & kAlignMask);
++ p->alignPriceCount++;
++ }
++ }
++ p->reps[3] = p->reps[2];
++ p->reps[2] = p->reps[1];
++ p->reps[1] = p->reps[0];
++ p->reps[0] = pos;
++ p->matchPriceCount++;
++ }
++ }
++ p->additionalOffset -= len;
++ nowPos32 += len;
++ if (p->additionalOffset == 0)
++ {
++ UInt32 processed;
++ if (!p->fastMode)
++ {
++ if (p->matchPriceCount >= (1 << 7))
++ FillDistancesPrices(p);
++ if (p->alignPriceCount >= kAlignTableSize)
++ FillAlignPrices(p);
++ }
++ if (p->matchFinder.GetNumAvailableBytes(p->matchFinderObj) == 0)
++ break;
++ processed = nowPos32 - startPos32;
++ if (useLimits)
++ {
++ if (processed + kNumOpts + 300 >= maxUnpackSize ||
++ RangeEnc_GetProcessed(&p->rc) + kNumOpts * 2 >= maxPackSize)
++ break;
++ }
++ else if (processed >= (1 << 15))
++ {
++ p->nowPos64 += nowPos32 - startPos32;
++ return CheckErrors(p);
++ }
++ }
++ }
++ p->nowPos64 += nowPos32 - startPos32;
++ return Flush(p, nowPos32);
++}
++
++#define kBigHashDicLimit ((UInt32)1 << 24)
++
++static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32 keepWindowSize, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ UInt32 beforeSize = kNumOpts;
++ Bool btMode;
++ if (!RangeEnc_Alloc(&p->rc, alloc))
++ return SZ_ERROR_MEM;
++ btMode = (p->matchFinderBase.btMode != 0);
++ #ifndef _7ZIP_ST
++ p->mtMode = (p->multiThread && !p->fastMode && btMode);
++ #endif
++
++ {
++ unsigned lclp = p->lc + p->lp;
++ if (p->litProbs == 0 || p->saveState.litProbs == 0 || p->lclp != lclp)
++ {
++ LzmaEnc_FreeLits(p, alloc);
++ p->litProbs = (CLzmaProb *)alloc->Alloc(alloc, (0x300 << lclp) * sizeof(CLzmaProb));
++ p->saveState.litProbs = (CLzmaProb *)alloc->Alloc(alloc, (0x300 << lclp) * sizeof(CLzmaProb));
++ if (p->litProbs == 0 || p->saveState.litProbs == 0)
++ {
++ LzmaEnc_FreeLits(p, alloc);
++ return SZ_ERROR_MEM;
++ }
++ p->lclp = lclp;
++ }
++ }
++
++ p->matchFinderBase.bigHash = (p->dictSize > kBigHashDicLimit);
++
++ if (beforeSize + p->dictSize < keepWindowSize)
++ beforeSize = keepWindowSize - p->dictSize;
++
++ #ifndef _7ZIP_ST
++ if (p->mtMode)
++ {
++ RINOK(MatchFinderMt_Create(&p->matchFinderMt, p->dictSize, beforeSize, p->numFastBytes, LZMA_MATCH_LEN_MAX, allocBig));
++ p->matchFinderObj = &p->matchFinderMt;
++ MatchFinderMt_CreateVTable(&p->matchFinderMt, &p->matchFinder);
++ }
++ else
++ #endif
++ {
++ if (!MatchFinder_Create(&p->matchFinderBase, p->dictSize, beforeSize, p->numFastBytes, LZMA_MATCH_LEN_MAX, allocBig))
++ return SZ_ERROR_MEM;
++ p->matchFinderObj = &p->matchFinderBase;
++ MatchFinder_CreateVTable(&p->matchFinderBase, &p->matchFinder);
++ }
++ return SZ_OK;
++}
++
++void LzmaEnc_Init(CLzmaEnc *p)
++{
++ UInt32 i;
++ p->state = 0;
++ for (i = 0 ; i < LZMA_NUM_REPS; i++)
++ p->reps[i] = 0;
++
++ RangeEnc_Init(&p->rc);
++
++
++ for (i = 0; i < kNumStates; i++)
++ {
++ UInt32 j;
++ for (j = 0; j < LZMA_NUM_PB_STATES_MAX; j++)
++ {
++ p->isMatch[i][j] = kProbInitValue;
++ p->isRep0Long[i][j] = kProbInitValue;
++ }
++ p->isRep[i] = kProbInitValue;
++ p->isRepG0[i] = kProbInitValue;
++ p->isRepG1[i] = kProbInitValue;
++ p->isRepG2[i] = kProbInitValue;
++ }
++
++ {
++ UInt32 num = 0x300 << (p->lp + p->lc);
++ for (i = 0; i < num; i++)
++ p->litProbs[i] = kProbInitValue;
++ }
++
++ {
++ for (i = 0; i < kNumLenToPosStates; i++)
++ {
++ CLzmaProb *probs = p->posSlotEncoder[i];
++ UInt32 j;
++ for (j = 0; j < (1 << kNumPosSlotBits); j++)
++ probs[j] = kProbInitValue;
++ }
++ }
++ {
++ for (i = 0; i < kNumFullDistances - kEndPosModelIndex; i++)
++ p->posEncoders[i] = kProbInitValue;
++ }
++
++ LenEnc_Init(&p->lenEnc.p);
++ LenEnc_Init(&p->repLenEnc.p);
++
++ for (i = 0; i < (1 << kNumAlignBits); i++)
++ p->posAlignEncoder[i] = kProbInitValue;
++
++ p->optimumEndIndex = 0;
++ p->optimumCurrentIndex = 0;
++ p->additionalOffset = 0;
++
++ p->pbMask = (1 << p->pb) - 1;
++ p->lpMask = (1 << p->lp) - 1;
++}
++
++void LzmaEnc_InitPrices(CLzmaEnc *p)
++{
++ if (!p->fastMode)
++ {
++ FillDistancesPrices(p);
++ FillAlignPrices(p);
++ }
++
++ p->lenEnc.tableSize =
++ p->repLenEnc.tableSize =
++ p->numFastBytes + 1 - LZMA_MATCH_LEN_MIN;
++ LenPriceEnc_UpdateTables(&p->lenEnc, 1 << p->pb, p->ProbPrices);
++ LenPriceEnc_UpdateTables(&p->repLenEnc, 1 << p->pb, p->ProbPrices);
++}
++
++static SRes LzmaEnc_AllocAndInit(CLzmaEnc *p, UInt32 keepWindowSize, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ UInt32 i;
++ for (i = 0; i < (UInt32)kDicLogSizeMaxCompress; i++)
++ if (p->dictSize <= ((UInt32)1 << i))
++ break;
++ p->distTableSize = i * 2;
++
++ p->finished = False;
++ p->result = SZ_OK;
++ RINOK(LzmaEnc_Alloc(p, keepWindowSize, alloc, allocBig));
++ LzmaEnc_Init(p);
++ LzmaEnc_InitPrices(p);
++ p->nowPos64 = 0;
++ return SZ_OK;
++}
++
++static SRes LzmaEnc_Prepare(CLzmaEncHandle pp, ISeqOutStream *outStream, ISeqInStream *inStream,
++ ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ p->matchFinderBase.stream = inStream;
++ p->needInit = 1;
++ p->rc.outStream = outStream;
++ return LzmaEnc_AllocAndInit(p, 0, alloc, allocBig);
++}
++
++SRes LzmaEnc_PrepareForLzma2(CLzmaEncHandle pp,
++ ISeqInStream *inStream, UInt32 keepWindowSize,
++ ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ p->matchFinderBase.stream = inStream;
++ p->needInit = 1;
++ return LzmaEnc_AllocAndInit(p, keepWindowSize, alloc, allocBig);
++}
++
++static void LzmaEnc_SetInputBuf(CLzmaEnc *p, const Byte *src, SizeT srcLen)
++{
++ p->matchFinderBase.directInput = 1;
++ p->matchFinderBase.bufferBase = (Byte *)src;
++ p->matchFinderBase.directInputRem = srcLen;
++}
++
++SRes LzmaEnc_MemPrepare(CLzmaEncHandle pp, const Byte *src, SizeT srcLen,
++ UInt32 keepWindowSize, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ LzmaEnc_SetInputBuf(p, src, srcLen);
++ p->needInit = 1;
++
++ return LzmaEnc_AllocAndInit(p, keepWindowSize, alloc, allocBig);
++}
++
++void LzmaEnc_Finish(CLzmaEncHandle pp)
++{
++ #ifndef _7ZIP_ST
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ if (p->mtMode)
++ MatchFinderMt_ReleaseStream(&p->matchFinderMt);
++ #else
++ pp = pp;
++ #endif
++}
++
++typedef struct
++{
++ ISeqOutStream funcTable;
++ Byte *data;
++ SizeT rem;
++ Bool overflow;
++} CSeqOutStreamBuf;
++
++static size_t MyWrite(void *pp, const void *data, size_t size)
++{
++ CSeqOutStreamBuf *p = (CSeqOutStreamBuf *)pp;
++ if (p->rem < size)
++ {
++ size = p->rem;
++ p->overflow = True;
++ }
++ memcpy(p->data, data, size);
++ p->rem -= size;
++ p->data += size;
++ return size;
++}
++
++
++UInt32 LzmaEnc_GetNumAvailableBytes(CLzmaEncHandle pp)
++{
++ const CLzmaEnc *p = (CLzmaEnc *)pp;
++ return p->matchFinder.GetNumAvailableBytes(p->matchFinderObj);
++}
++
++const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle pp)
++{
++ const CLzmaEnc *p = (CLzmaEnc *)pp;
++ return p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - p->additionalOffset;
++}
++
++SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit,
++ Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32 *unpackSize)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ UInt64 nowPos64;
++ SRes res;
++ CSeqOutStreamBuf outStream;
++
++ outStream.funcTable.Write = MyWrite;
++ outStream.data = dest;
++ outStream.rem = *destLen;
++ outStream.overflow = False;
++
++ p->writeEndMark = False;
++ p->finished = False;
++ p->result = SZ_OK;
++
++ if (reInit)
++ LzmaEnc_Init(p);
++ LzmaEnc_InitPrices(p);
++ nowPos64 = p->nowPos64;
++ RangeEnc_Init(&p->rc);
++ p->rc.outStream = &outStream.funcTable;
++
++ res = LzmaEnc_CodeOneBlock(p, True, desiredPackSize, *unpackSize);
++
++ *unpackSize = (UInt32)(p->nowPos64 - nowPos64);
++ *destLen -= outStream.rem;
++ if (outStream.overflow)
++ return SZ_ERROR_OUTPUT_EOF;
++
++ return res;
++}
++
++static SRes LzmaEnc_Encode2(CLzmaEnc *p, ICompressProgress *progress)
++{
++ SRes res = SZ_OK;
++
++ #ifndef _7ZIP_ST
++ Byte allocaDummy[0x300];
++ int i = 0;
++ for (i = 0; i < 16; i++)
++ allocaDummy[i] = (Byte)i;
++ #endif
++
++ for (;;)
++ {
++ res = LzmaEnc_CodeOneBlock(p, False, 0, 0);
++ if (res != SZ_OK || p->finished != 0)
++ break;
++ if (progress != 0)
++ {
++ res = progress->Progress(progress, p->nowPos64, RangeEnc_GetProcessed(&p->rc));
++ if (res != SZ_OK)
++ {
++ res = SZ_ERROR_PROGRESS;
++ break;
++ }
++ }
++ }
++ LzmaEnc_Finish(p);
++ return res;
++}
++
++SRes LzmaEnc_Encode(CLzmaEncHandle pp, ISeqOutStream *outStream, ISeqInStream *inStream, ICompressProgress *progress,
++ ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ RINOK(LzmaEnc_Prepare(pp, outStream, inStream, alloc, allocBig));
++ return LzmaEnc_Encode2((CLzmaEnc *)pp, progress);
++}
++
++SRes LzmaEnc_WriteProperties(CLzmaEncHandle pp, Byte *props, SizeT *size)
++{
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++ int i;
++ UInt32 dictSize = p->dictSize;
++ if (*size < LZMA_PROPS_SIZE)
++ return SZ_ERROR_PARAM;
++ *size = LZMA_PROPS_SIZE;
++ props[0] = (Byte)((p->pb * 5 + p->lp) * 9 + p->lc);
++
++ for (i = 11; i <= 30; i++)
++ {
++ if (dictSize <= ((UInt32)2 << i))
++ {
++ dictSize = (2 << i);
++ break;
++ }
++ if (dictSize <= ((UInt32)3 << i))
++ {
++ dictSize = (3 << i);
++ break;
++ }
++ }
++
++ for (i = 0; i < 4; i++)
++ props[1 + i] = (Byte)(dictSize >> (8 * i));
++ return SZ_OK;
++}
++
++SRes LzmaEnc_MemEncode(CLzmaEncHandle pp, Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
++ int writeEndMark, ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ SRes res;
++ CLzmaEnc *p = (CLzmaEnc *)pp;
++
++ CSeqOutStreamBuf outStream;
++
++ LzmaEnc_SetInputBuf(p, src, srcLen);
++
++ outStream.funcTable.Write = MyWrite;
++ outStream.data = dest;
++ outStream.rem = *destLen;
++ outStream.overflow = False;
++
++ p->writeEndMark = writeEndMark;
++
++ p->rc.outStream = &outStream.funcTable;
++ res = LzmaEnc_MemPrepare(pp, src, srcLen, 0, alloc, allocBig);
++ if (res == SZ_OK)
++ res = LzmaEnc_Encode2(p, progress);
++
++ *destLen -= outStream.rem;
++ if (outStream.overflow)
++ return SZ_ERROR_OUTPUT_EOF;
++ return res;
++}
++
++SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
++ const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeEndMark,
++ ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig)
++{
++ CLzmaEnc *p = (CLzmaEnc *)LzmaEnc_Create(alloc);
++ SRes res;
++ if (p == 0)
++ return SZ_ERROR_MEM;
++
++ res = LzmaEnc_SetProps(p, props);
++ if (res == SZ_OK)
++ {
++ res = LzmaEnc_WriteProperties(p, propsEncoded, propsSize);
++ if (res == SZ_OK)
++ res = LzmaEnc_MemEncode(p, dest, destLen, src, srcLen,
++ writeEndMark, progress, alloc, allocBig);
++ }
++
++ LzmaEnc_Destroy(p, alloc, allocBig);
++ return res;
++}
+
+Property changes on: third_party/lzma_sdk/LzmaEnc.c
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/Alloc.c
+===================================================================
+--- third_party/lzma_sdk/Alloc.c (revision 0)
++++ third_party/lzma_sdk/Alloc.c (revision 0)
+@@ -0,0 +1,127 @@
++/* Alloc.c -- Memory allocation functions
++2008-09-24
++Igor Pavlov
++Public domain */
++
++#ifdef _WIN32
++#include <windows.h>
++#endif
++#include <stdlib.h>
++
++#include "Alloc.h"
++
++/* #define _SZ_ALLOC_DEBUG */
++
++/* use _SZ_ALLOC_DEBUG to debug alloc/free operations */
++#ifdef _SZ_ALLOC_DEBUG
++#include <stdio.h>
++int g_allocCount = 0;
++int g_allocCountMid = 0;
++int g_allocCountBig = 0;
++#endif
++
++void *MyAlloc(size_t size)
++{
++ if (size == 0)
++ return 0;
++ #ifdef _SZ_ALLOC_DEBUG
++ {
++ void *p = malloc(size);
++ fprintf(stderr, "\nAlloc %10d bytes, count = %10d, addr = %8X", size, g_allocCount++, (unsigned)p);
++ return p;
++ }
++ #else
++ return malloc(size);
++ #endif
++}
++
++void MyFree(void *address)
++{
++ #ifdef _SZ_ALLOC_DEBUG
++ if (address != 0)
++ fprintf(stderr, "\nFree; count = %10d, addr = %8X", --g_allocCount, (unsigned)address);
++ #endif
++ free(address);
++}
++
++#ifdef _WIN32
++
++void *MidAlloc(size_t size)
++{
++ if (size == 0)
++ return 0;
++ #ifdef _SZ_ALLOC_DEBUG
++ fprintf(stderr, "\nAlloc_Mid %10d bytes; count = %10d", size, g_allocCountMid++);
++ #endif
++ return VirtualAlloc(0, size, MEM_COMMIT, PAGE_READWRITE);
++}
++
++void MidFree(void *address)
++{
++ #ifdef _SZ_ALLOC_DEBUG
++ if (address != 0)
++ fprintf(stderr, "\nFree_Mid; count = %10d", --g_allocCountMid);
++ #endif
++ if (address == 0)
++ return;
++ VirtualFree(address, 0, MEM_RELEASE);
++}
++
++#ifndef MEM_LARGE_PAGES
++#undef _7ZIP_LARGE_PAGES
++#endif
++
++#ifdef _7ZIP_LARGE_PAGES
++SIZE_T g_LargePageSize = 0;
++typedef SIZE_T (WINAPI *GetLargePageMinimumP)();
++#endif
++
++void SetLargePageSize()
++{
++ #ifdef _7ZIP_LARGE_PAGES
++ SIZE_T size = 0;
++ GetLargePageMinimumP largePageMinimum = (GetLargePageMinimumP)
++ GetProcAddress(GetModuleHandle(TEXT("kernel32.dll")), "GetLargePageMinimum");
++ if (largePageMinimum == 0)
++ return;
++ size = largePageMinimum();
++ if (size == 0 || (size & (size - 1)) != 0)
++ return;
++ g_LargePageSize = size;
++ #endif
++}
++
++
++void *BigAlloc(size_t size)
++{
++ if (size == 0)
++ return 0;
++ #ifdef _SZ_ALLOC_DEBUG
++ fprintf(stderr, "\nAlloc_Big %10d bytes; count = %10d", size, g_allocCountBig++);
++ #endif
++
++ #ifdef _7ZIP_LARGE_PAGES
++ if (g_LargePageSize != 0 && g_LargePageSize <= (1 << 30) && size >= (1 << 18))
++ {
++ void *res = VirtualAlloc(0, (size + g_LargePageSize - 1) & (~(g_LargePageSize - 1)),
++ MEM_COMMIT | MEM_LARGE_PAGES, PAGE_READWRITE);
++ if (res != 0)
++ return res;
++ }
++ #endif
++ return VirtualAlloc(0, size, MEM_COMMIT, PAGE_READWRITE);
++}
++
++void BigFree(void *address)
++{
++ #ifdef _SZ_ALLOC_DEBUG
++ if (address != 0)
++ fprintf(stderr, "\nFree_Big; count = %10d", --g_allocCountBig);
++ #endif
++
++ if (address == 0)
++ return;
++ VirtualFree(address, 0, MEM_RELEASE);
++}
++
++#endif
+
+Property changes on: third_party/lzma_sdk/Alloc.c
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzmaLib.c
+===================================================================
+--- third_party/lzma_sdk/LzmaLib.c (revision 0)
++++ third_party/lzma_sdk/LzmaLib.c (revision 0)
+@@ -0,0 +1,46 @@
++/* LzmaLib.c -- LZMA library wrapper
++2008-08-05
++Igor Pavlov
++Public domain */
++
++#include "LzmaEnc.h"
++#include "LzmaDec.h"
++#include "Alloc.h"
++#include "LzmaLib.h"
++
++static void *SzAlloc(void *p, size_t size) { p = p; return MyAlloc(size); }
++static void SzFree(void *p, void *address) { p = p; MyFree(address); }
++static ISzAlloc g_Alloc = { SzAlloc, SzFree };
++
++MY_STDAPI LzmaCompress(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t srcLen,
++ unsigned char *outProps, size_t *outPropsSize,
++ int level, /* 0 <= level <= 9, default = 5 */
++ unsigned dictSize, /* use (1 << N) or (3 << N). 4 KB < dictSize <= 128 MB */
++ int lc, /* 0 <= lc <= 8, default = 3 */
++ int lp, /* 0 <= lp <= 4, default = 0 */
++ int pb, /* 0 <= pb <= 4, default = 2 */
++ int fb, /* 5 <= fb <= 273, default = 32 */
++ int numThreads /* 1 or 2, default = 2 */
++)
++{
++ CLzmaEncProps props;
++ LzmaEncProps_Init(&props);
++ props.level = level;
++ props.dictSize = dictSize;
++ props.lc = lc;
++ props.lp = lp;
++ props.pb = pb;
++ props.fb = fb;
++ props.numThreads = numThreads;
++
++ return LzmaEncode(dest, destLen, src, srcLen, &props, outProps, outPropsSize, 0,
++ NULL, &g_Alloc, &g_Alloc);
++}
++
++
++MY_STDAPI LzmaUncompress(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t *srcLen,
++ const unsigned char *props, size_t propsSize)
++{
++ ELzmaStatus status;
++ return LzmaDecode(dest, destLen, src, srcLen, props, (unsigned)propsSize, LZMA_FINISH_ANY, &status, &g_Alloc);
++}
+
+Property changes on: third_party/lzma_sdk/LzmaLib.c
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
+Index: third_party/lzma_sdk/LzFind.h
+===================================================================
+--- third_party/lzma_sdk/LzFind.h (revision 0)
++++ third_party/lzma_sdk/LzFind.h (revision 0)
+@@ -0,0 +1,115 @@
++/* LzFind.h -- Match finder for LZ algorithms
++2009-04-22 : Igor Pavlov : Public domain */
++
++#ifndef __LZ_FIND_H
++#define __LZ_FIND_H
++
++#include "Types.h"
++
++#ifdef __cplusplus
++extern "C" {
++#endif
++
++typedef UInt32 CLzRef;
++
++typedef struct _CMatchFinder
++{
++ Byte *buffer;
++ UInt32 pos;
++ UInt32 posLimit;
++ UInt32 streamPos;
++ UInt32 lenLimit;
++
++ UInt32 cyclicBufferPos;
++ UInt32 cyclicBufferSize; /* it must be = (historySize + 1) */
++
++ UInt32 matchMaxLen;
++ CLzRef *hash;
++ CLzRef *son;
++ UInt32 hashMask;
++ UInt32 cutValue;
++
++ Byte *bufferBase;
++ ISeqInStream *stream;
++ int streamEndWasReached;
++
++ UInt32 blockSize;
++ UInt32 keepSizeBefore;
++ UInt32 keepSizeAfter;
++
++ UInt32 numHashBytes;
++ int directInput;
++ size_t directInputRem;
++ int btMode;
++ int bigHash;
++ UInt32 historySize;
++ UInt32 fixedHashSize;
++ UInt32 hashSizeSum;
++ UInt32 numSons;
++ SRes result;
++ UInt32 crc[256];
++} CMatchFinder;
++
++#define Inline_MatchFinder_GetPointerToCurrentPos(p) ((p)->buffer)
++#define Inline_MatchFinder_GetIndexByte(p, index) ((p)->buffer[(Int32)(index)])
++
++#define Inline_MatchFinder_GetNumAvailableBytes(p) ((p)->streamPos - (p)->pos)
++
++int MatchFinder_NeedMove(CMatchFinder *p);
++Byte *MatchFinder_GetPointerToCurrentPos(CMatchFinder *p);
++void MatchFinder_MoveBlock(CMatchFinder *p);
++void MatchFinder_ReadIfRequired(CMatchFinder *p);
++
++void MatchFinder_Construct(CMatchFinder *p);
++
++/* Conditions:
++ historySize <= 3 GB
++ keepAddBufferBefore + matchMaxLen + keepAddBufferAfter < 511MB
++*/
++int MatchFinder_Create(CMatchFinder *p, UInt32 historySize,
++ UInt32 keepAddBufferBefore, UInt32 matchMaxLen, UInt32 keepAddBufferAfter,
++ ISzAlloc *alloc);
++void MatchFinder_Free(CMatchFinder *p, ISzAlloc *alloc);
++void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, UInt32 numItems);
++void MatchFinder_ReduceOffsets(CMatchFinder *p, UInt32 subValue);
++
++UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, const Byte *buffer, CLzRef *son,
++ UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue,
++ UInt32 *distances, UInt32 maxLen);
++
++/*
++Conditions:
++ Mf_GetNumAvailableBytes_Func must be called before each Mf_GetMatchLen_Func.
++ Mf_GetPointerToCurrentPos_Func's result must be used only before any other function
++*/
++
++typedef void (*Mf_Init_Func)(void *object);
++typedef Byte (*Mf_GetIndexByte_Func)(void *object, Int32 index);
++typedef UInt32 (*Mf_GetNumAvailableBytes_Func)(void *object);
++typedef const Byte * (*Mf_GetPointerToCurrentPos_Func)(void *object);
++typedef UInt32 (*Mf_GetMatches_Func)(void *object, UInt32 *distances);
++typedef void (*Mf_Skip_Func)(void *object, UInt32);
++
++typedef struct _IMatchFinder
++{
++ Mf_Init_Func Init;
++ Mf_GetIndexByte_Func GetIndexByte;
++ Mf_GetNumAvailableBytes_Func GetNumAvailableBytes;
++ Mf_GetPointerToCurrentPos_Func GetPointerToCurrentPos;
++ Mf_GetMatches_Func GetMatches;
++ Mf_Skip_Func Skip;
++} IMatchFinder;
++
++void MatchFinder_CreateVTable(CMatchFinder *p, IMatchFinder *vTable);
++
++void MatchFinder_Init(CMatchFinder *p);
++UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances);
++UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances);
++void Bt3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num);
++void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num);
++
++#ifdef __cplusplus
++}
++#endif
++
++#endif
+
+Property changes on: third_party/lzma_sdk/LzFind.h
+___________________________________________________________________
+Added: svn:eol-style
+ + LF
+
diff --git a/src/com/google/typography/font/compression/AdvWidth.java b/src/com/google/typography/font/compression/AdvWidth.java
new file mode 100644
index 0000000..a95a1cd
--- /dev/null
+++ b/src/com/google/typography/font/compression/AdvWidth.java
@@ -0,0 +1,27 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.typography.font.sfntly.Font;
+import com.google.typography.font.sfntly.Tag;
+import com.google.typography.font.sfntly.data.WritableFontData;
+import com.google.typography.font.sfntly.table.core.HorizontalMetricsTable;
+
+/**
+ * Extract just advance widths from hmtx table.
+ *
+ * @author raph@google.com (Raph Levien)
+ */
+public class AdvWidth {
+ public static WritableFontData encode(Font font) {
+ HorizontalMetricsTable hmtx = font.getTable(Tag.hmtx);
+ int nMetrics = hmtx.numberOfHMetrics();
+ WritableFontData result = WritableFontData.createWritableFontData(nMetrics * 2);
+ for (int i = 0; i < nMetrics; i++) {
+ result.writeShort(i * 2, hmtx.hMetricAdvanceWidth(i));
+ }
+ return result;
+ }
+}
diff --git a/src/com/google/typography/font/compression/CmapEncoder.java b/src/com/google/typography/font/compression/CmapEncoder.java
new file mode 100644
index 0000000..f461831
--- /dev/null
+++ b/src/com/google/typography/font/compression/CmapEncoder.java
@@ -0,0 +1,82 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.typography.font.sfntly.Font;
+import com.google.typography.font.sfntly.Tag;
+import com.google.typography.font.sfntly.table.core.CMap;
+import com.google.typography.font.sfntly.table.core.CMapTable;
+import com.google.typography.font.sfntly.table.core.MaximumProfileTable;
+
+import java.io.ByteArrayOutputStream;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * @author raph@google.com (Raph Levien)
+ *
+ * Experimental CMap encoder, based primarily on writing the _inverse_ encoding.
+ */
+public class CmapEncoder {
+ public static byte[] encode(Font font) {
+ int nGlyphs = font.<MaximumProfileTable>getTable(Tag.maxp).numGlyphs();
+ CMapTable cmapTable = font.getTable(Tag.cmap);
+ CMap cmap = getBestCMap(cmapTable);
+ Map<Integer, Integer> invEncoding = Maps.newHashMap();
+ List<Integer> exceptions = Lists.newArrayList();
+ for (Integer i : cmap) {
+ int glyphId = cmap.glyphId(i);
+ if (invEncoding.containsKey(glyphId)) {
+ exceptions.add(i);
+ exceptions.add(glyphId);
+ } else {
+ invEncoding.put(glyphId, i);
+ }
+ }
+ ByteArrayOutputStream os = new ByteArrayOutputStream();
+ int last = -1;
+ for (int i = 0; i < nGlyphs; i++) {
+ if (invEncoding.containsKey(i)) {
+ int value = invEncoding.get(i);
+ int delta = value - last;
+ writeVShort(os, delta);
+ last = value;
+ } else {
+ writeVShort(os, 0);
+ }
+ }
+ for (int i : exceptions) {
+ writeVShort(os, i);
+ }
+ return os.toByteArray();
+ }
+
+ private static CMap getBestCMap(CMapTable cmapTable) {
+ for (CMap cmap : cmapTable) {
+ if (cmap.format() == CMap.CMapFormat.Format12.value()) {
+ return cmap;
+ }
+ }
+ for (CMap cmap : cmapTable) {
+ if (cmap.format() == CMap.CMapFormat.Format4.value()) {
+ return cmap;
+ }
+ }
+ return null;
+ }
+
+ // A simple signed varint encoding
+ static void writeVShort(ByteArrayOutputStream os, int value) {
+ if (value >= 0x2000 || value < -0x2000) {
+ os.write((byte)(0x80 | ((value >> 14) & 0x7f)));
+ }
+ if (value >= 0x40 || value < -0x40) {
+ os.write((byte)(0x80 | ((value >> 7) & 0x7f)));
+ }
+ os.write((byte)(value & 0x7f));
+ }
+}
diff --git a/src/com/google/typography/font/compression/Command.java b/src/com/google/typography/font/compression/Command.java
new file mode 100644
index 0000000..6b11f31
--- /dev/null
+++ b/src/com/google/typography/font/compression/Command.java
@@ -0,0 +1,72 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.common.io.ByteStreams;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+/**
+ * This is a simple wrapper to run a commandline as a pipe. It's not
+ * particularly efficient, robust, or featureful.
+ */
+public class Command {
+
+ private class StreamConsumer extends Thread {
+ private final InputStream is;
+ private byte[] result = null;
+
+ public StreamConsumer(InputStream is) {
+ this.is = is;
+ }
+
+ @Override
+ public void run() {
+ try {
+ result = ByteStreams.toByteArray(is);
+ } catch (IOException e) {
+ // TODO: handle this well
+ }
+ }
+
+ public byte[] getResult() {
+ return result;
+ }
+ }
+
+ private final ProcessBuilder processBuilder;
+
+ public Command(String[] args) {
+ processBuilder = new ProcessBuilder(args);
+ }
+
+ public CommandResult execute(byte[] input) throws CommandException {
+ Process process;
+ try {
+ process = processBuilder.start();
+ } catch (IOException e) {
+ throw new CommandException("exec failed");
+ }
+ StreamConsumer sc = new StreamConsumer(process.getInputStream());
+ sc.start();
+ OutputStream stdin = process.getOutputStream();
+ try {
+ stdin.write(input);
+ stdin.close();
+ } catch (IOException e) {
+ throw new CommandException("error writing input");
+ }
+ try {
+ process.waitFor();
+ sc.join();
+ } catch (InterruptedException e) {
+ throw new CommandException("interrupted");
+ }
+ return new CommandResult(sc.getResult());
+ }
+}
+
diff --git a/src/com/google/typography/font/compression/CommandException.java b/src/com/google/typography/font/compression/CommandException.java
new file mode 100644
index 0000000..9f9125f
--- /dev/null
+++ b/src/com/google/typography/font/compression/CommandException.java
@@ -0,0 +1,12 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+public class CommandException extends Exception {
+ public CommandException(String message) {
+ super(message);
+ }
+}
+
diff --git a/src/com/google/typography/font/compression/CommandResult.java b/src/com/google/typography/font/compression/CommandResult.java
new file mode 100644
index 0000000..4f1e787
--- /dev/null
+++ b/src/com/google/typography/font/compression/CommandResult.java
@@ -0,0 +1,18 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+public final class CommandResult {
+ private final byte[] stdout;
+
+ public CommandResult(byte[] stdout) {
+ this.stdout = stdout;
+ }
+
+ public byte[] getStdout() {
+ return stdout;
+ }
+}
+
diff --git a/src/com/google/typography/font/compression/CompressLzma.java b/src/com/google/typography/font/compression/CompressLzma.java
new file mode 100644
index 0000000..5b749c1
--- /dev/null
+++ b/src/com/google/typography/font/compression/CompressLzma.java
@@ -0,0 +1,22 @@
+// Copyright 2012 Google Inc. All Rights Reserved.
+
+package com.google.typography.font.compression;
+
+/**
+ * @author raph@google.com (Raph Levien)
+ */
+public class CompressLzma {
+ // This is currently implemented by shelling out to a command line helper,
+ // which is fine for research purposes, but obviously problematic for
+ // production.
+ public static byte[] compress(byte[] input) {
+ try {
+ String[] args = {"/usr/bin/lzma"};
+ CommandResult result = new Command(args).execute(input);
+ return result.getStdout();
+ } catch (CommandException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+}
diff --git a/src/com/google/typography/font/compression/CompressionRunner.java b/src/com/google/typography/font/compression/CompressionRunner.java
new file mode 100644
index 0000000..e7263f0
--- /dev/null
+++ b/src/com/google/typography/font/compression/CompressionRunner.java
@@ -0,0 +1,283 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import com.google.common.io.Files;
+import com.google.typography.font.sfntly.Font;
+import com.google.typography.font.sfntly.FontFactory;
+import com.google.typography.font.sfntly.Tag;
+import com.google.typography.font.sfntly.data.ReadableFontData;
+import com.google.typography.font.tools.conversion.eot.EOTWriter;
+import com.google.typography.font.tools.conversion.eot.HdmxEncoder;
+import com.google.typography.font.tools.conversion.woff.WoffWriter;
+
+import java.io.ByteArrayOutputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map.Entry;
+import java.util.Set;
+
+/**
+ * A command-line tool for running different experimental compression code over
+ * a corpus of fonts, and gathering statistics, particularly compression
+ * efficiency.
+ *
+ * This is not intended to be production code, and for certain codecs it will
+ * shell out to helper binaries, which is fine for the purposes of gathering
+ * statistics, but obviously not much else.
+ *
+ * @author raph@google.com (Raph Levien)
+ */
+public class CompressionRunner {
+ // private static final boolean DEBUG = false;
+
+ public static void main(String[] args) throws IOException {
+ boolean generateOutput = false;
+ List<String> descs = Lists.newArrayList();
+ String baseline = "gzip";
+
+ List<String> filenames = Lists.newArrayList();
+ for (int i = 0; i < args.length; i++) {
+ if (args[i].charAt(0) == '-') {
+ if (args[i].equals("-o")) {
+ generateOutput = true;
+ } else if (args[i].equals("-x")) {
+ descs.add(args[i + 1]);
+ i++;
+ } else if (args[i].equals("-b")) {
+ baseline = args[i + 1];
+ i++;
+ }
+ } else {
+ filenames.add(args[i]);
+ }
+ }
+
+ // String baseline = "glyf/triplet,code,push:lzma";
+ // String baseline = "glyf/cbbox,triplet,code,push:hdmx:lzma";
+ // descs.add("woff2");
+ if (descs.isEmpty()) {
+ descs.add("glyf/cbbox,triplet,code,reslice:woff2");
+ }
+ runTest(filenames, baseline, descs, generateOutput);
+ }
+
+ private static void runTest(List<String> filenames, String baseline, List<String> descs,
+ boolean generateOutput) throws IOException {
+ PrintWriter o = new PrintWriter(System.out);
+ List<StatsCollector> stats = Lists.newArrayList();
+ for (int i = 0; i < descs.size(); i++) {
+ stats.add(new StatsCollector());
+ }
+ FontFactory fontFactory = FontFactory.getInstance();
+ o.println("<html>");
+ for (String filename : filenames) {
+ byte[] bytes = Files.toByteArray(new File(filename));
+ Font font = fontFactory.loadFonts(bytes)[0];
+ byte[] baselineResult = runExperiment(font, baseline);
+ o.printf("<!-- %s: baseline %d bytes", new File(filename).getName(), baselineResult.length);
+ for (int i = 0; i < descs.size(); i++) {
+ byte[] expResult = runExperiment(font, descs.get(i));
+ if (generateOutput) {
+ String newFilename = filename;
+ if (newFilename.endsWith(".ttf")) {
+ newFilename = newFilename.substring(0, newFilename.length() - 4);
+ }
+ newFilename += ".woff2";
+ Files.write(expResult, new File(newFilename));
+ }
+ double percent = 100. * expResult.length / baselineResult.length;
+ stats.get(i).addStat(percent);
+ o.printf(", %c %.2f%%", 'A' + i, percent);
+ }
+ o.printf(" -->\n");
+ }
+ stats.get(0).chartHeader(o, descs.size());
+ for (int i = 0; i < descs.size(); i++) {
+ stats.get(i).chartData(o, i + 1);
+ }
+ stats.get(0).chartEnd(o);
+ o.printf("<p>baseline: %s</p>\n", baseline);
+ for (int i = 0; i < descs.size(); i++) {
+ StatsCollector sc = stats.get(i);
+ o.printf("<p>%c: %s: median %f, mean %f</p>\n",
+ 'A' + i, descs.get(i), sc.median(), sc.mean());
+ }
+ stats.get(0).chartFooter(o);
+ o.close();
+ }
+
+ private static Font.Builder stripTags(Font srcFont, Set<Integer> removeTags) {
+ FontFactory fontFactory = FontFactory.getInstance();
+ Font.Builder fontBuilder = fontFactory.newFontBuilder();
+
+ for (Integer tag : srcFont.tableMap().keySet()) {
+ if (!removeTags.contains(tag)) {
+ fontBuilder.newTableBuilder(tag, srcFont.getTable(tag).readFontData());
+ }
+ }
+ return fontBuilder;
+ }
+
+ private static Font.Builder preprocessMtxGlyf(Font srcFont, String options) {
+ Font.Builder fontBuilder = stripTags(srcFont, ImmutableSet.<Integer>of());
+ GlyfEncoder glyfEncoder = new GlyfEncoder(options);
+ glyfEncoder.encode(srcFont);
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'g', 'l', 'z', '1'}),
+ glyfEncoder.getGlyfBytes());
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'l', 'o', 'c', 'z'}),
+ glyfEncoder.getLocaBytes());
+ if (!Arrays.asList(options.split(",")).contains("reslice")) {
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'g', 'l', 'z', '2'}),
+ glyfEncoder.getCodeBytes());
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'g', 'l', 'z', '3'}),
+ glyfEncoder.getPushBytes());
+ }
+ return fontBuilder;
+ }
+
+ private static Font.Builder preprocessHmtx(Font srcFont) {
+ Font.Builder fontBuilder = stripTags(srcFont, ImmutableSet.of(Tag.hmtx));
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'h', 'm', 't', 'z'}),
+ toBytes(AdvWidth.encode(srcFont)));
+ return fontBuilder;
+ }
+
+ private static Font.Builder preprocessHdmx(Font srcFont) {
+ Font.Builder fontBuilder = stripTags(srcFont, ImmutableSet.of(Tag.hdmx));
+ if (srcFont.hasTable(Tag.hdmx)) {
+ addTableBytes(fontBuilder, Tag.hdmx, toBytes(new HdmxEncoder().encode(srcFont)));
+ }
+ return fontBuilder;
+ }
+
+ private static Font.Builder preprocessCmap(Font srcFont) {
+ Font.Builder fontBuilder = stripTags(srcFont, ImmutableSet.of(Tag.cmap));
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'c', 'm', 'a', 'z'}),
+ CmapEncoder.encode(srcFont));
+ return fontBuilder;
+ }
+
+ private static Font.Builder preprocessKern(Font srcFont) {
+ Font.Builder fontBuilder = stripTags(srcFont, ImmutableSet.of(Tag.kern));
+ if (srcFont.hasTable(Tag.kern)) {
+ addTableBytes(fontBuilder, Tag.intValue(new byte[] {'k', 'e', 'r', 'z'}),
+ toBytes(KernEncoder.encode(srcFont)));
+ }
+ return fontBuilder;
+ }
+
+ private static void addTableBytes(Font.Builder fontBuilder, int tag, byte[] contents) {
+ fontBuilder.newTableBuilder(tag, ReadableFontData.createReadableFontData(contents));
+ }
+
+ private static byte[] fontToBytes(FontFactory fontFactory, Font font) {
+ try {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ fontFactory.serializeFont(font, baos);
+ return baos.toByteArray();
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ private static byte[] toBytes(ReadableFontData rfd) {
+ byte[] result = new byte[rfd.length()];
+ rfd.readBytes(0, result, 0, rfd.length());
+ return result;
+ }
+
+ // This is currently implemented by shelling out to a command line helper,
+ // which is fine for research purposes, but obviously problematic for
+ // production.
+ private static byte[] compressBzip2(byte[] input) {
+ try {
+ String[] args = {"/bin/bzip2"};
+ CommandResult result = new Command(args).execute(input);
+ return result.getStdout();
+ } catch (CommandException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ /**
+ * Does one experimental compression on a font, using the string to guide what
+ * gets done.
+ *
+ * @param srcFont Source font
+ * @param desc experiment description string; the exact format is probably
+ * still evolving
+ * @return serialization of compressed font
+ * @throws IOException
+ */
+ private static byte[] runExperiment(Font srcFont, String desc) throws IOException {
+ Font font = srcFont;
+ FontFactory fontFactory = FontFactory.getInstance();
+ String[] pieces = desc.split(":");
+ boolean keepDsig = false;
+ for (int i = 0; i < pieces.length - 1; i++) {
+ String[] piece = pieces[i].split("/");
+ String cmd = piece[0];
+ if (cmd.equals("glyf")) {
+ font = preprocessMtxGlyf(font, piece.length > 1 ? piece[1] : "").build();
+ } else if (cmd.equals("hmtx")) {
+ font = preprocessHmtx(font).build();
+ } else if (cmd.equals("hdmx")) {
+ font = preprocessHdmx(font).build();
+ } else if (cmd.equals("cmap")) {
+ font = preprocessCmap(font).build();
+ } else if (cmd.equals("kern")) {
+ font = preprocessKern(font).build();
+ } else if (cmd.equals("keepdsig")) {
+ keepDsig = true;
+ } else if (cmd.equals("strip")) {
+ Set<Integer> removeTags = Sets.newTreeSet();
+ for (String tag : piece[1].split(",")) {
+ removeTags.add(Tag.intValue(tag.getBytes()));
+ }
+ font = stripTags(font, removeTags).build();
+ }
+ }
+ if (!keepDsig) {
+ font = stripTags(font, ImmutableSet.of(Tag.DSIG)).build();
+ }
+ String last = pieces[pieces.length - 1];
+ String[] lastPieces = last.split("/");
+ String lastBase = lastPieces[0];
+ String lastArgs = lastPieces.length > 1 ? lastPieces[1] : "";
+ if (!lastBase.equals("woff2")) {
+ Set<Integer> tagsToStrip = Sets.newHashSet();
+ for (Entry<Integer, Integer> mapping : Woff2Writer.getTransformMap().entrySet()) {
+ if (font.hasTable(mapping.getValue())) {
+ tagsToStrip.add(mapping.getKey());
+ }
+ }
+ font = stripTags(font, tagsToStrip).build();
+ }
+ byte[] result = null;
+ if (lastBase.equals("gzip")) {
+ result = GzipUtil.deflate(fontToBytes(fontFactory, font));
+ } else if (lastBase.equals("lzma")) {
+ result = CompressLzma.compress(fontToBytes(fontFactory, font));
+ } else if (lastBase.equals("bzip2")) {
+ result = compressBzip2(fontToBytes(fontFactory, font));
+ } else if (lastBase.equals("woff")) {
+ result = toBytes(new WoffWriter().convert(font));
+ } else if (lastBase.equals("woff2")) {
+ result = toBytes(new Woff2Writer(lastArgs).convert(srcFont, font));
+ } else if (lastBase.equals("eot")) {
+ result = toBytes(new EOTWriter(true).convert(font));
+ } else if (lastBase.equals("uncomp")) {
+ result = fontToBytes(fontFactory, font);
+ }
+ return result;
+ }
+}
diff --git a/src/com/google/typography/font/compression/GlyfEncoder.java b/src/com/google/typography/font/compression/GlyfEncoder.java
new file mode 100644
index 0000000..ebaa80e
--- /dev/null
+++ b/src/com/google/typography/font/compression/GlyfEncoder.java
@@ -0,0 +1,481 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.typography.font.sfntly.Font;
+import com.google.typography.font.sfntly.Tag;
+import com.google.typography.font.sfntly.data.ReadableFontData;
+import com.google.typography.font.sfntly.table.core.FontHeaderTable;
+import com.google.typography.font.sfntly.table.truetype.CompositeGlyph;
+import com.google.typography.font.sfntly.table.truetype.Glyph;
+import com.google.typography.font.sfntly.table.truetype.GlyphTable;
+import com.google.typography.font.sfntly.table.truetype.LocaTable;
+import com.google.typography.font.sfntly.table.truetype.SimpleGlyph;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * @author Raph Levien
+ *
+ * Implementation of compression of CTF glyph data, as per sections 5.6-5.10 and 6 of the spec.
+ * This is a hacked-up version with a number of options, for experimenting.
+ */
+public class GlyfEncoder {
+
+ private final ByteArrayOutputStream nContourStream;
+ private final ByteArrayOutputStream nPointsStream;
+ private final ByteArrayOutputStream flagBytesStream;
+ private final ByteArrayOutputStream compositeStream;
+ private final ByteArrayOutputStream bboxStream;
+ private final ByteArrayOutputStream glyfStream;
+ private final ByteArrayOutputStream pushStream;
+ private final ByteArrayOutputStream codeStream;
+ private final boolean sbbox;
+ private final boolean cbbox;
+ private final boolean code;
+ private final boolean triplet;
+ private final boolean doPush;
+ private final boolean doHop;
+ private final boolean push2byte;
+ private final boolean reslice;
+
+ private int nGlyphs;
+ private byte[] bboxBitmap;
+ private FontHeaderTable.IndexToLocFormat indexFmt;
+
+ public GlyfEncoder(String options) {
+ glyfStream = new ByteArrayOutputStream();
+ pushStream = new ByteArrayOutputStream();
+ codeStream = new ByteArrayOutputStream();
+ nContourStream = new ByteArrayOutputStream();
+ nPointsStream = new ByteArrayOutputStream();
+ flagBytesStream = new ByteArrayOutputStream();
+ compositeStream = new ByteArrayOutputStream();
+ bboxStream = new ByteArrayOutputStream();
+ boolean sbbox = false;
+ boolean cbbox = false;
+ boolean code = false;
+ boolean triplet = false;
+ boolean doPush = false;
+ boolean reslice = false;
+ boolean doHop = false;
+ boolean push2byte = false;
+ for (String option : options.split(",")) {
+ if (option.equals("sbbox")) {
+ sbbox = true;
+ } else if (option.equals("cbbox")) {
+ cbbox = true;
+ } else if (option.equals("code")) {
+ code = true;
+ } else if (option.equals("triplet")) {
+ triplet = true;
+ } else if (option.equals("push")) {
+ doPush = true;
+ } else if (option.equals("hop")) {
+ doHop = true;
+ } else if (option.equals("push2byte")) {
+ push2byte = true;
+ } else if (option.equals("reslice")) {
+ reslice = true;
+ }
+ }
+ this.sbbox = sbbox;
+ this.cbbox = cbbox;
+ this.code = code;
+ this.triplet = triplet;
+ this.doPush = doPush;
+ this.doHop = doHop;
+ this.push2byte = push2byte;
+ this.reslice = reslice;
+ }
+
+ public void encode(Font sourceFont) {
+ FontHeaderTable head = sourceFont.getTable(Tag.head);
+ indexFmt = head.indexToLocFormat();
+ LocaTable loca = sourceFont.getTable(Tag.loca);
+ nGlyphs = loca.numGlyphs();
+ GlyphTable glyf = sourceFont.getTable(Tag.glyf);
+ bboxBitmap = new byte[((nGlyphs + 31) >> 5) << 2];
+
+ for (int glyphId = 0; glyphId < nGlyphs; glyphId++) {
+ int sourceOffset = loca.glyphOffset(glyphId);
+ int length = loca.glyphLength(glyphId);
+ Glyph glyph = glyf.glyph(sourceOffset, length);
+ writeGlyph(glyphId, glyph);
+ }
+ }
+
+ private void writeGlyph(int glyphId, Glyph glyph) {
+ try {
+ if (glyph == null || glyph.dataLength() == 0) {
+ writeNContours(0);
+ } else if (glyph instanceof SimpleGlyph) {
+ writeSimpleGlyph(glyphId, (SimpleGlyph)glyph);
+ } else if (glyph instanceof CompositeGlyph) {
+ writeCompositeGlyph(glyphId, (CompositeGlyph)glyph);
+ }
+ } catch (IOException e) {
+ throw new RuntimeException("unexpected IOException writing glyph data", e);
+ }
+ }
+
+ private void writeInstructions(Glyph glyph) throws IOException{
+ if (doPush) {
+ splitPush(glyph);
+ } else {
+ int pushCount = 0;
+ int codeSize = glyph.instructionSize();
+ if (!reslice) {
+ write255UShort(glyfStream, pushCount);
+ }
+ write255UShort(glyfStream, codeSize);
+ if (codeSize > 0) {
+ if (code) {
+ glyph.instructions().copyTo(codeStream);
+ } else {
+ glyph.instructions().copyTo(glyfStream);
+ }
+ }
+ }
+ }
+
+ private void writeSimpleGlyph(int glyphId, SimpleGlyph glyph) throws IOException {
+ int numContours = glyph.numberOfContours();
+ writeNContours(numContours);
+ if (sbbox) {
+ writeBbox(glyphId, glyph);
+ }
+ // TODO: check that bbox matches, write bbox if not
+ for (int i = 0; i < numContours; i++) {
+ if (reslice) {
+ write255UShort(nPointsStream, glyph.numberOfPoints(i));
+ } else {
+ write255UShort(glyfStream, glyph.numberOfPoints(i) - (i == 0 ? 1 : 0));
+ }
+ }
+ ByteArrayOutputStream os = new ByteArrayOutputStream();
+ int lastX = 0;
+ int lastY = 0;
+ for (int i = 0; i < numContours; i++) {
+ int numPoints = glyph.numberOfPoints(i);
+ for (int j = 0; j < numPoints; j++) {
+ int x = glyph.xCoordinate(i, j);
+ int y = glyph.yCoordinate(i, j);
+ int dx = x - lastX;
+ int dy = y - lastY;
+ if (triplet) {
+ writeTriplet(os, glyph.onCurve(i, j), dx, dy);
+ } else {
+ writeVShort(os, dx * 2 + (glyph.onCurve(i, j) ? 1 : 0));
+ writeVShort(os, dy);
+ }
+ lastX = x;
+ lastY = y;
+ }
+ }
+ os.writeTo(glyfStream);
+ if (numContours > 0) {
+ writeInstructions(glyph);
+ }
+ }
+
+ private void writeCompositeGlyph(int glyphId, CompositeGlyph glyph) throws IOException {
+ boolean haveInstructions = false;
+ writeNContours(-1);
+ if (cbbox) {
+ writeBbox(glyphId, glyph);
+ }
+ ByteArrayOutputStream outStream = reslice ? compositeStream : glyfStream;
+ for (int i = 0; i < glyph.numGlyphs(); i++) {
+ int flags = glyph.flags(i);
+ writeUShort(outStream, flags);
+ haveInstructions = (flags & CompositeGlyph.FLAG_WE_HAVE_INSTRUCTIONS) != 0;
+ writeUShort(outStream, glyph.glyphIndex(i));
+ if ((flags & CompositeGlyph.FLAG_ARG_1_AND_2_ARE_WORDS) == 0) {
+ outStream.write(glyph.argument1(i));
+ outStream.write(glyph.argument2(i));
+ } else {
+ writeUShort(outStream, glyph.argument1(i));
+ writeUShort(outStream, glyph.argument2(i));
+ }
+ if (glyph.transformationSize(i) != 0) {
+ try {
+ outStream.write(glyph.transformation(i));
+ } catch (IOException e) {
+ }
+ }
+ }
+ if (haveInstructions) {
+ writeInstructions(glyph);
+ }
+ }
+
+ private void writeNContours(int nContours) {
+ if (reslice) {
+ writeUShort(nContourStream, nContours);
+ } else {
+ writeUShort(nContours);
+ }
+ }
+
+ private void writeBbox(int glyphId, Glyph glyph) {
+ if (reslice) {
+ bboxBitmap[glyphId >> 3] |= 0x80 >> (glyphId & 7);
+ }
+ ByteArrayOutputStream outStream = reslice ? bboxStream : glyfStream;
+ writeUShort(outStream, glyph.xMin());
+ writeUShort(outStream, glyph.yMin());
+ writeUShort(outStream, glyph.xMax());
+ writeUShort(outStream, glyph.yMax());
+ }
+
+ private void writeUShort(ByteArrayOutputStream os, int value) {
+ os.write(value >> 8);
+ os.write(value & 255);
+ }
+
+ private void writeUShort(int value) {
+ writeUShort(glyfStream, value);
+ }
+
+ private void writeLong(OutputStream os, int value) throws IOException {
+ os.write((value >> 24) & 255);
+ os.write((value >> 16) & 255);
+ os.write((value >> 8) & 255);
+ os.write(value & 255);
+ }
+
+ // As per 6.1.1 of spec
+ // visible for testing
+ static void write255UShort(ByteArrayOutputStream os, int value) {
+ if (value < 0) {
+ throw new IllegalArgumentException();
+ }
+ if (value < 253) {
+ os.write((byte)value);
+ } else if (value < 506) {
+ os.write(255);
+ os.write((byte)(value - 253));
+ } else if (value < 762) {
+ os.write(254);
+ os.write((byte)(value - 506));
+ } else {
+ os.write(253);
+ os.write((byte)(value >> 8));
+ os.write((byte)(value & 0xff));
+ }
+ }
+
+ // As per 6.1.1 of spec
+ // visible for testing
+ static void write255Short(OutputStream os, int value) throws IOException {
+ int absValue = Math.abs(value);
+ if (value < 0) {
+ // spec is unclear about whether words should be signed. This code is conservative, but we
+ // can test once the implementation is working.
+ os.write(250);
+ }
+ if (absValue < 250) {
+ os.write((byte)absValue);
+ } else if (absValue < 500) {
+ os.write(255);
+ os.write((byte)(absValue - 250));
+ } else if (absValue < 756) {
+ os.write(254);
+ os.write((byte)(absValue - 500));
+ } else {
+ os.write(253);
+ os.write((byte)(absValue >> 8));
+ os.write((byte)(absValue & 0xff));
+ }
+ }
+
+ // A simple signed varint encoding
+ static void writeVShort(ByteArrayOutputStream os, int value) {
+ if (value >= 0x2000 || value < -0x2000) {
+ os.write((byte)(0x80 | ((value >> 14) & 0x7f)));
+ }
+ if (value >= 0x40 || value < -0x40) {
+ os.write((byte)(0x80 | ((value >> 7) & 0x7f)));
+ }
+ os.write((byte)(value & 0x7f));
+ }
+
+ // As in section 5.11 of the spec
+ // visible for testing
+ void writeTriplet(OutputStream os, boolean onCurve, int x, int y) throws IOException {
+ int absX = Math.abs(x);
+ int absY = Math.abs(y);
+ int onCurveBit = onCurve ? 0 : 128;
+ int xSignBit = (x < 0) ? 0 : 1;
+ int ySignBit = (y < 0) ? 0 : 1;
+ int xySignBits = xSignBit + 2 * ySignBit;
+ ByteArrayOutputStream flagStream = reslice ? flagBytesStream : glyfStream;
+
+ if (x == 0 && absY < 1280) {
+ flagStream.write(onCurveBit + ((absY & 0xf00) >> 7) + ySignBit);
+ os.write(absY & 0xff);
+ } else if (y == 0 && absX < 1280) {
+ flagStream.write(onCurveBit + 10 + ((absX & 0xf00) >> 7) + xSignBit);
+ os.write(absX & 0xff);
+ } else if (absX < 65 && absY < 65) {
+ flagStream.write(onCurveBit + 20 + ((absX - 1) & 0x30) + (((absY - 1) & 0x30) >> 2) +
+ xySignBits);
+ os.write((((absX - 1) & 0xf) << 4) | ((absY - 1) & 0xf));
+ } else if (absX < 769 && absY < 769) {
+ flagStream.write(onCurveBit + 84 + 12 * (((absX - 1) & 0x300) >> 8) +
+ (((absY - 1) & 0x300) >> 6) + xySignBits);
+ os.write((absX - 1) & 0xff);
+ os.write((absY - 1) & 0xff);
+ } else if (absX < 4096 && absY < 4096) {
+ flagStream.write(onCurveBit + 120 + xySignBits);
+ os.write(absX >> 4);
+ os.write(((absX & 0xf) << 4) | (absY >> 8));
+ os.write(absY & 0xff);
+ } else {
+ flagStream.write(onCurveBit + 124 + xySignBits);
+ os.write(absX >> 8);
+ os.write(absX & 0xff);
+ os.write(absY >> 8);
+ os.write(absY & 0xff);
+ }
+ }
+
+ /**
+ * Split the instructions into a push sequence and the remainder of the instructions.
+ * Writes both streams, and the counts to the glyfStream.
+ *
+ * @param glyph
+ */
+ private void splitPush(Glyph glyph) throws IOException {
+ int instrSize = glyph.instructionSize();
+ ReadableFontData data = glyph.instructions();
+ int i = 0;
+ List<Integer> result = new ArrayList<Integer>();
+ // All push sequences are at least two bytes, make sure there's enough room
+ while (i + 1 < instrSize) {
+ int ix = i;
+ int instr = data.readUByte(ix++);
+ int n = 0;
+ int size = 0;
+ if (instr == 0x40 || instr == 0x41) {
+ // NPUSHB, NPUSHW
+ n = data.readUByte(ix++);
+ size = (instr & 1) + 1;
+ } else if (instr >= 0xB0 && instr < 0xC0) {
+ // PUSHB, PUSHW
+ n = 1 + (instr & 7);
+ size = ((instr & 8) >> 3) + 1;
+ } else {
+ break;
+ }
+ if (i + size * n > instrSize) {
+ // This is a broken font, and a potential buffer overflow, but in the interest
+ // of preserving the original, we just put the broken instruction sequence in
+ // the stream.
+ break;
+ }
+ for (int j = 0; j < n; j++) {
+ if (size == 1) {
+ result.add(data.readUByte(ix));
+ } else {
+ result.add(data.readShort(ix));
+ }
+ ix += size;
+ }
+ i = ix;
+ }
+ int pushCount = result.size();
+ int codeSize = instrSize - i;
+ write255UShort(glyfStream, pushCount);
+ write255UShort(glyfStream, codeSize);
+ encodePushSequence(pushStream, result);
+ if (codeSize > 0) {
+ data.slice(i).copyTo(codeStream);
+ }
+ }
+
+ // As per section 6.2.2 of the spec
+ private void encodePushSequence(ByteArrayOutputStream os, List<Integer> data) throws IOException {
+ int n = data.size();
+ int hopSkip = 0;
+ for (int i = 0; i < n; i++) {
+ if ((hopSkip & 1) == 0) {
+ int val = data.get(i);
+ if (doHop && hopSkip == 0 && i >= 2 &&
+ i + 2 < n && val == data.get(i - 2) && val == data.get(i + 2)) {
+ if (i + 4 < n && val == data.get(i + 4)) {
+ // Hop4 code
+ os.write(252);
+ hopSkip = 0x14;
+ } else {
+ // Hop3 code
+ os.write(251);
+ hopSkip = 4;
+ }
+ } else {
+ if (push2byte) {
+ // Measure relative effectiveness of 255Short literal encoding vs 2-byte ushort.
+ writeUShort(os, data.get(i));
+ } else {
+ write255Short(os, data.get(i));
+ }
+ }
+ }
+ hopSkip >>= 1;
+ }
+ }
+
+ public byte[] getGlyfBytes() {
+ if (reslice) {
+ ByteArrayOutputStream newStream = new ByteArrayOutputStream();
+ try {
+ // Pack all the glyf streams in a sensible way
+ writeLong(newStream, 0); // version
+ writeUShort(newStream, nGlyphs);
+ writeUShort(newStream, indexFmt.value());
+ writeLong(newStream, nContourStream.size());
+ writeLong(newStream, nPointsStream.size());
+ writeLong(newStream, flagBytesStream.size());
+ writeLong(newStream, glyfStream.size());
+ writeLong(newStream, compositeStream.size());
+ writeLong(newStream, bboxBitmap.length + bboxStream.size());
+ writeLong(newStream, codeStream.size());
+ System.out.printf("stream sizes = %d %d %d %d %d %d %d\n",
+ nContourStream.size(), nPointsStream.size(), flagBytesStream.size(), glyfStream.size(),
+ compositeStream.size(), bboxStream.size(), codeStream.size());
+ nContourStream.writeTo(newStream);
+ nPointsStream.writeTo(newStream);
+ flagBytesStream.writeTo(newStream);
+ glyfStream.writeTo(newStream);
+ compositeStream.writeTo(newStream);
+ newStream.write(bboxBitmap);
+ bboxStream.writeTo(newStream);
+ codeStream.writeTo(newStream);
+ } catch (IOException e) {
+ throw new RuntimeException("Can't happen, world must have come to end", e);
+ }
+ return newStream.toByteArray();
+ } else {
+ return glyfStream.toByteArray();
+ }
+ }
+
+ public byte[] getPushBytes() {
+ return pushStream.toByteArray();
+ }
+
+ public byte[] getCodeBytes() {
+ return codeStream.toByteArray();
+ }
+
+ public byte[] getLocaBytes() {
+ return new byte[]{ };
+ }
+}
diff --git a/src/com/google/typography/font/compression/GzipUtil.java b/src/com/google/typography/font/compression/GzipUtil.java
new file mode 100644
index 0000000..af04b8a
--- /dev/null
+++ b/src/com/google/typography/font/compression/GzipUtil.java
@@ -0,0 +1,28 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.zip.Deflater;
+import java.util.zip.DeflaterOutputStream;
+
+/**
+ * Simple utility for GZIP compression
+ */
+public class GzipUtil {
+ public static byte[] deflate(byte[] bytes) {
+ try {
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DeflaterOutputStream dos = new DeflaterOutputStream(baos, new Deflater());
+ dos.write(bytes, 0, bytes.length);
+ dos.close();
+ return baos.toByteArray();
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ }
+}
+
diff --git a/src/com/google/typography/font/compression/KernEncoder.java b/src/com/google/typography/font/compression/KernEncoder.java
new file mode 100644
index 0000000..c3df4c7
--- /dev/null
+++ b/src/com/google/typography/font/compression/KernEncoder.java
@@ -0,0 +1,37 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.typography.font.sfntly.Font;
+import com.google.typography.font.sfntly.Tag;
+import com.google.typography.font.sfntly.data.ReadableFontData;
+import com.google.typography.font.sfntly.data.WritableFontData;
+import com.google.typography.font.sfntly.table.Table;
+
+/**
+ * @author raph@google.com (Raph Levien)
+ *
+ * Encoder for "kern" table. This probably won't go in the spec because an even more
+ * effective technique would be to do class kerning in the GDEF tables, but, even so, I wanted
+ * to capture the stats.
+ */
+public class KernEncoder {
+ public static WritableFontData encode(Font font) {
+ Table kernTable = font.getTable(Tag.kern);
+ ReadableFontData data = kernTable.readFontData();
+ WritableFontData newData = WritableFontData.createWritableFontData(data.size());
+ data.copyTo(newData);
+ if (data.readUShort(0) == 0 && data.readUShort(4) == 0) {
+ int base = 18;
+ int nPairs = data.readUShort(10);
+ for (int i = 0; i < nPairs; i++) {
+ newData.writeUShort(base + i * 2, data.readUShort(base + i * 6));
+ newData.writeUShort(base + nPairs * 2 + i * 2, data.readUShort(base + i * 6 + 2));
+ newData.writeUShort(base + nPairs * 4 + i * 2, data.readUShort(base + i * 6 + 4));
+ }
+ }
+ return newData;
+ }
+}
diff --git a/src/com/google/typography/font/compression/StatsCollector.java b/src/com/google/typography/font/compression/StatsCollector.java
new file mode 100644
index 0000000..6eb7154
--- /dev/null
+++ b/src/com/google/typography/font/compression/StatsCollector.java
@@ -0,0 +1,92 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.common.collect.Lists;
+
+import java.io.PrintWriter;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Class for gathering up stats, for summarizing and graphing.
+ *
+ * @author raph@google.com (Raph Levien)
+ */
+public class StatsCollector {
+ private final List<Double> values;
+
+ public StatsCollector() {
+ values = Lists.newArrayList();
+ }
+
+ public void addStat(double value) {
+ values.add(value);
+ }
+
+ public double mean() {
+ double sum = 0;
+ for (Double value : values) {
+ sum += value;
+ }
+ return sum / values.size();
+ }
+
+ public double median() {
+ Collections.sort(values);
+ int length = values.size();
+ if (length % 2 == 1) {
+ return values.get((length - 1) / 2);
+ } else {
+ return 0.5 * (values.get(length / 2 - 1) + values.get(length / 2));
+ }
+ }
+
+ // Need to print <html> before calling this method
+ public void chartHeader(PrintWriter o, int n) {
+ o.println("<head>");
+ o.println("<script type='text/javascript' src='https://www.google.com/jsapi'></script>");
+ o.println("<script type='text/javascript'>");
+ o.println("google.load('visualization', '1', {packages:['corechart']});");
+ o.println("google.setOnLoadCallback(drawChart);");
+ o.println("function drawChart() {");
+ o.println(" var data = new google.visualization.DataTable()");
+ o.println(" data.addColumn('string', 'Font');");
+ if (n == 1) {
+ o.println(" data.addColumn('number', 'Ratio');");
+ } else {
+ for (int i = 0; i < n; i++) {
+ o.printf(" data.addColumn('number', 'Ratio %c');\n", 'A' + i);
+ }
+ }
+ o.printf(" data.addRows(%d);\n", values.size());
+ }
+
+ public void chartData(PrintWriter o, int ix) {
+ Collections.sort(values);
+ int length = values.size();
+ for (int i = 0; i < length; i++) {
+ o.printf(" data.setValue(%d, %d, %f);\n", i, ix, values.get(i));
+ }
+ }
+
+ public void chartEnd(PrintWriter o) {
+ o.println(" var chart = new google.visualization.LineChart(document.getElementById("
+ + "'chart_div'));");
+ o.println(" chart.draw(data, {width:700, height:400, title: 'Compression ratio'});");
+ o.println("}");
+ o.println("</script>");
+ o.println("</head>");
+
+ o.println();
+ o.println("<body>");
+ o.println("<div id='chart_div'></div>");
+ // TODO: split so we can get content into the HTML
+ }
+ public void chartFooter(PrintWriter o) {
+ o.println("</body>");
+ o.println("</html>");
+ }
+}
diff --git a/src/com/google/typography/font/compression/TestCommand.java b/src/com/google/typography/font/compression/TestCommand.java
new file mode 100644
index 0000000..d527f6b
--- /dev/null
+++ b/src/com/google/typography/font/compression/TestCommand.java
@@ -0,0 +1,28 @@
+// Copyright 2011 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import java.io.IOException;
+
+/**
+ * A simple test for the command mechanism. Quick and dirty run with:
+ * java -cp 'build/classes:lib/guava-11.0.1.jar' com/google/typography/font/compression/TestCommand
+ */
+public class TestCommand {
+ public static void main(String[] args) throws IOException {
+ String[] commandArgs = {"/usr/bin/lzma"};
+ byte[] input = new byte[16384];
+ try {
+ CommandResult result = new Command(commandArgs).execute(input);
+ byte[] output = result.getStdout();
+ for (int i = 0; i < output.length; i++) {
+ System.out.printf("%02x\n", output[i] & 0xff);
+ }
+ } catch (CommandException e) {
+ e.printStackTrace();
+ };
+ }
+}
+
diff --git a/src/com/google/typography/font/compression/Woff2Writer.java b/src/com/google/typography/font/compression/Woff2Writer.java
new file mode 100644
index 0000000..7f51d0f
--- /dev/null
+++ b/src/com/google/typography/font/compression/Woff2Writer.java
@@ -0,0 +1,352 @@
+// Copyright 2012 Google Inc. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package com.google.typography.font.compression;
+
+import com.google.common.collect.ImmutableBiMap;
+import com.google.common.collect.Lists;
+import com.google.typography.font.sfntly.Font;
+import com.google.typography.font.sfntly.Tag;
+import com.google.typography.font.sfntly.data.WritableFontData;
+import com.google.typography.font.sfntly.table.Table;
+import com.google.typography.font.sfntly.table.core.FontHeaderTable;
+
+import java.util.List;
+import java.util.TreeSet;
+
+public class Woff2Writer {
+ private static final long SIGNATURE = 0x774f4632;
+ private static final int WOFF2_HEADER_SIZE = 44;
+ private static final int TABLE_ENTRY_SIZE = 5 * 4;
+ private static final int FLAG_CONTINUE_STREAM = 1 << 4;
+ private static final int FLAG_APPLY_TRANSFORM = 1 << 5;
+
+ private final CompressionType compressionType;
+ private final boolean longForm;
+
+ Woff2Writer(String args) {
+ CompressionType compressionType = CompressionType.NONE;
+ boolean longForm = false;
+ for (String arg : args.split(",")) {
+ if ("lzma".equals(arg)) {
+ compressionType = CompressionType.LZMA;
+ } else if ("gzip".equals(arg)) {
+ compressionType = CompressionType.GZIP;
+ } else if ("short".equals(arg)) {
+ longForm = false;
+ } else if ("long".equals(arg)) {
+ longForm = true;
+ }
+ }
+ this.compressionType = compressionType;
+ this.longForm = longForm;
+ }
+
+ private static ImmutableBiMap<Integer, Integer> TRANSFORM_MAP = ImmutableBiMap.of(
+ Tag.glyf, Tag.intValue(new byte[] {'g', 'l', 'z', '1'}),
+ Tag.loca, Tag.intValue(new byte[] {'l', 'o', 'c', 'z'})
+ );
+
+ public static ImmutableBiMap<Integer, Integer> getTransformMap() {
+ return TRANSFORM_MAP;
+ }
+
+ private static ImmutableBiMap<Integer, Integer> KNOWN_TABLES =
+ new ImmutableBiMap.Builder<Integer, Integer>()
+ .put(Tag.intValue(new byte[] {'c', 'm', 'a', 'p'}), 0)
+ .put(Tag.intValue(new byte[] {'h', 'e', 'a', 'd'}), 1)
+ .put(Tag.intValue(new byte[] {'h', 'h', 'e', 'a'}), 2)
+ .put(Tag.intValue(new byte[] {'h', 'm', 't', 'x'}), 3)
+ .put(Tag.intValue(new byte[] {'m', 'a', 'x', 'p'}), 4)
+ .put(Tag.intValue(new byte[] {'n', 'a', 'm', 'e'}), 5)
+ .put(Tag.intValue(new byte[] {'O', 'S', '/', '2'}), 6)
+ .put(Tag.intValue(new byte[] {'p', 'o', 's', 't'}), 7)
+ .put(Tag.intValue(new byte[] {'c', 'v', 't', ' '}), 8)
+ .put(Tag.intValue(new byte[] {'f', 'p', 'g', 'm'}), 9)
+ .put(Tag.intValue(new byte[] {'g', 'l', 'y', 'f'}), 10)
+ .put(Tag.intValue(new byte[] {'l', 'o', 'c', 'a'}), 11)
+ .put(Tag.intValue(new byte[] {'p', 'r', 'e', 'p'}), 12)
+ .put(Tag.intValue(new byte[] {'C', 'F', 'F', ' '}), 13)
+ .put(Tag.intValue(new byte[] {'V', 'O', 'R', 'G'}), 14)
+ .put(Tag.intValue(new byte[] {'E', 'B', 'D', 'T'}), 15)
+ .put(Tag.intValue(new byte[] {'E', 'B', 'L', 'C'}), 16)
+ .put(Tag.intValue(new byte[] {'g', 'a', 's', 'p'}), 17)
+ .put(Tag.intValue(new byte[] {'h', 'd', 'm', 'x'}), 18)
+ .put(Tag.intValue(new byte[] {'k', 'e', 'r', 'n'}), 19)
+ .put(Tag.intValue(new byte[] {'L', 'T', 'S', 'H'}), 20)
+ .put(Tag.intValue(new byte[] {'P', 'C', 'L', 'T'}), 21)
+ .put(Tag.intValue(new byte[] {'V', 'D', 'M', 'X'}), 22)
+ .put(Tag.intValue(new byte[] {'v', 'h', 'e', 'a'}), 23)
+ .put(Tag.intValue(new byte[] {'v', 'm', 't', 'x'}), 24)
+ .put(Tag.intValue(new byte[] {'B', 'A', 'S', 'E'}), 25)
+ .put(Tag.intValue(new byte[] {'G', 'D', 'E', 'F'}), 26)
+ .put(Tag.intValue(new byte[] {'G', 'P', 'O', 'S'}), 27)
+ .put(Tag.intValue(new byte[] {'G', 'S', 'U', 'B'}), 28)
+ .build();
+
+ public WritableFontData convert(Font srcFont, Font font) {
+ List<TableDirectoryEntry> entries = createTableDirectoryEntries(font);
+ int size = computeCompressedFontSize(entries);
+ WritableFontData writableFontData = WritableFontData.createWritableFontData(size);
+ int index = 0;
+ FontHeaderTable head = font.getTable(Tag.head);
+ index += writeWoff2Header(writableFontData, entries, font.sfntVersion(), size,
+ head.fontRevision());
+ System.out.printf("Wrote header, index = %d\n", index);
+ index += writeDirectory(writableFontData, index, entries);
+ System.out.printf("Wrote directory, index = %d\n", index);
+ index += writeTables(writableFontData, index, entries);
+ System.out.printf("Wrote tables, index = %d\n", index);
+ return writableFontData;
+ }
+
+ private List<TableDirectoryEntry> createTableDirectoryEntries(Font font) {
+ List<TableDirectoryEntry> entries = Lists.newArrayList();
+ TreeSet<Integer> tags = new TreeSet<Integer>(font.tableMap().keySet());
+
+ for (int tag : tags) {
+ Table table = font.getTable(tag);
+ byte[] uncompressedBytes = bytesFromTable(table);
+ byte[] transformedBytes = null;
+ if (TRANSFORM_MAP.containsValue(tag)) {
+ // Don't store the intermediate transformed tables under the nonstandard tags.
+ continue;
+ }
+ if (TRANSFORM_MAP.containsKey(tag)) {
+ int transformedTag = TRANSFORM_MAP.get(tag);
+ Table transformedTable = font.getTable(transformedTag);
+ if (transformedTable != null) {
+ transformedBytes = bytesFromTable(transformedTable);
+ }
+ }
+ if (transformedBytes == null) {
+ entries.add(new TableDirectoryEntry(tag, uncompressedBytes, compressionType));
+ } else {
+ entries.add(new TableDirectoryEntry(tag, uncompressedBytes, transformedBytes,
+ FLAG_APPLY_TRANSFORM, compressionType));
+ }
+ }
+ return entries;
+ }
+
+ private byte[] bytesFromTable(Table table) {
+ int length = table.dataLength();
+ byte[] bytes = new byte[length];
+ table.readFontData().readBytes(0, bytes, 0, length);
+ return bytes;
+ }
+
+ private int writeWoff2Header(WritableFontData writableFontData,
+ List<TableDirectoryEntry> entries,
+ int flavor,
+ int length,
+ int version) {
+ int index = 0;
+ index += writableFontData.writeULong(index, SIGNATURE);
+ index += writableFontData.writeULong(index, flavor);
+ index += writableFontData.writeULong(index, length);
+ index += writableFontData.writeUShort(index, entries.size()); // numTables
+ index += writableFontData.writeUShort(index, 0); // reserved
+ int uncompressedFontSize = computeUncompressedSize(entries);
+ index += writableFontData.writeULong(index, uncompressedFontSize);
+ index += writableFontData.writeFixed(index, version);
+ index += writableFontData.writeULong(index, 0); // metaOffset
+ index += writableFontData.writeULong(index, 0); // metaLength
+ index += writableFontData.writeULong(index, 0); // metaOrigLength
+ index += writableFontData.writeULong(index, 0); // privOffset
+ index += writableFontData.writeULong(index, 0); // privLength
+ return index;
+ }
+
+ private int writeDirectory(WritableFontData writableFontData, int offset,
+ List<TableDirectoryEntry> entries) {
+ int directorySize = computeDirectoryLength(entries);
+ for (TableDirectoryEntry entry : entries) {
+ offset += entry.writeEntry(writableFontData, offset);
+ }
+ return directorySize;
+ }
+
+ private int writeTables(WritableFontData writableFontData, int offset,
+ List<TableDirectoryEntry> entries) {
+ int start = offset;
+ for (TableDirectoryEntry entry : entries) {
+ offset += entry.writeData(writableFontData, offset);
+ offset = align4(offset);
+ }
+ return offset - start;
+ }
+
+ private int computeDirectoryLength(List<TableDirectoryEntry> entries) {
+ if (longForm) {
+ return TABLE_ENTRY_SIZE * entries.size();
+ } else {
+ int size = 0;
+ for (TableDirectoryEntry entry : entries) {
+ size += entry.writeEntry(null, size);
+ }
+ return size;
+ }
+ }
+
+ private int align4(int value) {
+ return (value + 3) & -4;
+ }
+
+ private int computeUncompressedSize(List<TableDirectoryEntry> entries) {
+ int size = 20 + 16 * entries.size(); // sfnt header length
+ for (TableDirectoryEntry entry : entries) {
+ size += entry.getOrigLength();
+ size = align4(size);
+ }
+ return size;
+ }
+
+ private int computeCompressedFontSize(List<TableDirectoryEntry> entries) {
+ int fontSize = WOFF2_HEADER_SIZE;
+ fontSize += computeDirectoryLength(entries);
+ for (TableDirectoryEntry entry : entries) {
+ fontSize += entry.getCompLength();
+ fontSize = align4(fontSize);
+ }
+ return fontSize;
+ }
+
+ private enum CompressionType {
+ NONE, GZIP, LZMA
+ }
+
+ private static long flagsForCompression(CompressionType compressionType) {
+ switch (compressionType) {
+ case NONE:
+ return 0;
+ case GZIP:
+ return 1;
+ case LZMA:
+ return 2;
+ }
+ return 0;
+ }
+
+ private static byte[] compress(byte[] input, CompressionType compressionType) {
+ switch (compressionType) {
+ case NONE:
+ return input;
+ case GZIP:
+ return GzipUtil.deflate(input);
+ case LZMA:
+ return CompressLzma.compress(input);
+ }
+ return null;
+ }
+
+ // Note: if writableFontData is null, just return the size
+ private static int writeBase128(WritableFontData writableFontData, long value, int offset) {
+ int size = 1;
+ long tmpValue = value;
+ while (tmpValue >= 128) {
+ size += 1;
+ tmpValue = tmpValue >> 7;
+ }
+ for (int i = 0; i < size; i++) {
+ int b = (int)(value >> (7 * (size - i - 1))) & 0x7f;
+ if (i < size - 1) {
+ b |= 0x80;
+ }
+ if (writableFontData != null) {
+ writableFontData.writeByte(offset, (byte)b);
+ }
+ offset += 1;
+ }
+ return size;
+ }
+
+ private class TableDirectoryEntry {
+ private final long tag;
+ private final long flags;
+ private final long origLength;
+ private final long transformLength;
+ private final byte[] bytes;
+
+ // This is the constructor for tables that don't have transforms
+ public TableDirectoryEntry(long tag, byte[] uncompressedBytes,
+ CompressionType compressionType) {
+ this(tag, uncompressedBytes, uncompressedBytes, 0, compressionType);
+ }
+
+ public TableDirectoryEntry(long tag, byte[] uncompressedBytes, byte[] transformedBytes,
+ long transformFlags, CompressionType compressionType) {
+ byte[] compressedBytes = compress(transformedBytes, compressionType);
+ if (compressedBytes.length >= transformedBytes.length) {
+ compressedBytes = transformedBytes;
+ compressionType = CompressionType.NONE;
+ }
+ this.tag = tag;
+ this.flags = transformFlags | flagsForCompression(compressionType);
+ this.origLength = uncompressedBytes.length;
+ this.transformLength = transformedBytes.length;
+ this.bytes = compressedBytes;
+ }
+
+ public long getOrigLength() {
+ return origLength;
+ }
+
+ public long getCompLength() {
+ return bytes.length;
+ }
+
+ // Note: if writableFontData is null, just return the size
+ public int writeEntry(WritableFontData writableFontData, int offset) {
+ if (longForm) {
+ if (writableFontData != null) {
+ offset += writableFontData.writeULong(offset, tag);
+ offset += writableFontData.writeULong(offset, flags);
+ offset += writableFontData.writeULong(offset, getCompLength());
+ offset += writableFontData.writeULong(offset, transformLength);
+ offset += writableFontData.writeULong(offset, getOrigLength());
+ }
+ return TABLE_ENTRY_SIZE;
+ } else {
+ int start = offset;
+ int flag_byte = 0x1f;
+ if (KNOWN_TABLES.containsKey((int)tag)) {
+ flag_byte = KNOWN_TABLES.get((int)tag);
+ }
+ if ((flags & FLAG_APPLY_TRANSFORM) != 0) {
+ flag_byte |= 0x20;
+ }
+ if ((flags & FLAG_CONTINUE_STREAM) != 0) {
+ flag_byte |= 0xc0;
+ } else {
+ flag_byte |= (flags & 3) << 6;
+ }
+ if (writableFontData != null) {
+ System.out.printf("%d: tag = %08x, flag = %02x\n", offset, tag, flag_byte);
+ writableFontData.writeByte(offset, (byte)flag_byte);
+ }
+ offset += 1;
+ if ((flag_byte & 0x1f) == 0x1f) {
+ if (writableFontData != null) {
+ writableFontData.writeULong(offset, tag);
+ }
+ offset += 4;
+ }
+ offset += writeBase128(writableFontData, getOrigLength(), offset);
+ if ((flag_byte & 0x20) != 0) {
+ offset += writeBase128(writableFontData, transformLength, offset);
+ }
+ if ((flag_byte & 0xc0) == 0x40 || (flag_byte & 0xc0) == 0x80) {
+ offset += writeBase128(writableFontData, getCompLength(), offset);
+ }
+ return offset - start;
+ }
+ }
+
+ public int writeData(WritableFontData writableFontData, int offset) {
+ writableFontData.writeBytes(offset, bytes);
+ return bytes.length;
+ }
+ }
+}
diff --git a/woff2_header_dump.py b/woff2_header_dump.py
new file mode 100644
index 0000000..b352d50
--- /dev/null
+++ b/woff2_header_dump.py
@@ -0,0 +1,38 @@
+# Copyright (c) 2012 Google Inc. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+# This is a simple utility for dumping out the header of a compressed file, and
+# is suitable for doing spot checks of compressed. files. However, this only
+# implements the "long" form of the table directory.
+
+import struct
+import sys
+
+def dump_woff2_header(header):
+ header_values = struct.unpack('>IIIHHIHHIIIII', header[:44])
+ for i, key in enumerate([
+ 'signature',
+ 'flavor',
+ 'length',
+ 'numTables',
+ 'reserved',
+ 'totalSfntSize',
+ 'majorVersion',
+ 'minorVersion',
+ 'metaOffset',
+ 'metaOrigLength',
+ 'privOffset',
+ 'privLength']):
+ print key, header_values[i]
+ numTables = header_values[3]
+ for i in range(numTables):
+ entry = struct.unpack('>IIIII', header[44+20*i:44+20*(i+1)])
+ print '%08x %d %d %d %d' % entry
+
+def main():
+ header = file(sys.argv[1]).read()
+ dump_woff2_header(header)
+
+main()
+
