README.txt - platform/external/gemmlowp - Git at Google

 gemmlowp: a small self-contained low-precision GEMM library
 ===========================================================

 This is not a full linear algebra library, only a GEMM library: it only does
 general matrix multiplication ("GEMM").

 The meaning of "low precision" is detailed in this document:
   doc/low-precision.txt

 Some of the general design is explained in
   doc/design.txt


 Disclaimer
 ==========

 This is not an official Google product (experimental or otherwise), it is just
 code that happens to be owned by Google.


 Mailing list
 ============

 gemmlowp-related discussion, about either development or usage, is welcome
 on this Google Group (mailing list / forum):

   https://groups.google.com/forum/#!forum/gemmlowp


 Portability, target platforms/architectures
 ===========================================

 Should be portable to any platform with some C++11 and POSIX support,
 while we have optional optimized code paths for specific architectures.

 Required:
   C++11 (a small conservative subset of it)

 Required for some features:
   * Some POSIX interfaces:
     * pthreads (for multi-threaded operation and for profiling).
     * sysconf (for multi-threaded operation to detect number of cores;
                may be bypassed).

 Optional:
   Architecture-specific code paths use intrinsics or inline assembly.
   See "Architecture-specific optimized code paths" below.

 Architecture-specific optimized code paths
 ==========================================

 We have some optimized code paths for specific instruction sets.
 Some are written in inline assembly, some are written in C++ using
 intrinsics. Both GCC and Clang are supported.

 At the moment, we have a full set of optimized code paths (kernels,
 packing and unpacking paths) only for ARM NEON, supporting both
 ARMv7 (32bit) and ARMv8 (64bit).

 We also have a partial set of optimized code paths (only kernels
 at the moment) for Intel SSE. It supports both x86 and x86-64 but
 only targets SSE4. The lack of packing/unpacking code paths means
 that performance isn't optimal yet.

 Details of what it takes to make an efficient port of gemmlowp, namely
 writing a suitable GEMM kernel and accompanying packing code, are
 explained in this file:
   doc/kernels.txt


 Public interfaces
 =================

 1. gemmlowp public interface
 ----------------------------

   gemmlowp's main public interface is in the public/ subdirectory. The
   header to include is
       public/gemmlowp.h.
   This is a headers-only library, so there is nothing to link to.

 2. EightBitIntGemm standard interface
 -------------------------------------

   Additionally, the eight_bit_int_gemm/ subdirectory provides an
   implementation of the standard EightBitIntGemm interface. The header
   to include is
     eight_bit_int_gemm/eight_bit_int_gemm.h
   This is *NOT* a headers-only library, users need to link to
     eight_bit_int_gemm/eight_bit_int_gemm.cc.
   The API is similar to the standard BLAS GEMM interface, and implements
   C = A * B. If the transpose flags for a matrix argument are false, its memory
   order is treated as column major, and row major if its true.


 Building
 ========

 Building by manually invoking your compiler
 -------------------------------------------

 Because gemmlowp is so simple, working with it involves only
 single-command-line compiler invokations. Therefore we expect that
 most people working with gemmlowp will either manually invoke their
 compiler, or write their own rules for their own preferred build
 system.

 Keep in mind (previous section) that gemmlowp itself is a pure-headers-only
 library so there is nothing to build, and the eight_bit_int_gemm library
 consists of a single eight_bit_int_gemm.cc file to build.

 For a Android gemmlowp development workflow, the scripts/ directory
 contains a script to build and run a program on an Android device:
   scripts/test-android.sh

 Building using Bazel
 --------------------

 That being said, we also maintain a Bazel BUILD system as part of
 gemmlowp. Its usage is not mandatory at all and is only one
 possible way that gemmlowp libraries and tests may be built. If
 you are interested, Bazel's home page is
   http://bazel.io/
 And you can get started with using Bazel to build gemmlowp targets
 by first creating an empty WORKSPACE file in a parent directory,
 for instance:

 $ cd gemmlowp/..  # change to parent directory containing gemmlowp/
 $ touch WORKSPACE # declare that to be our workspace root
 $ bazel build gemmlowp:all


 Testing
 =======

 Testing by manually building and running tests
 ----------------------------------------------

 The test/ directory contains unit tests. The primary unit test is
   test/test.cc
 Since it covers also the EightBitIntGemm interface, it needs to be
 linked against
   eight_bit_int_gemm/eight_bit_int_gemm.cc
 It also uses realistic data captured from a neural network run in
   test/test_data.cc

 Thus you'll want to pass the following list of source files to your
 compiler/linker:
   test/test.cc
   eight_bit_int_gemm/eight_bit_int_gemm.cc
   test/test_data.cc

 The scripts/ directory contains a script to build and run a program
 on an Android device:
   scripts/test-android.sh

 It expects the CXX environment variable to point to an Android toolchain's
 C++ compiler, and expects source files (and optionally, cflags) as
 command-line parameters. To build and run the above-mentioned main unit test,
 first set CXX e.g.:

 $ export CXX=/some/toolchains/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-g++

 Then run:

 $ ./scripts/test-android.sh \
 test/test.cc \
 eight_bit_int_gemm/eight_bit_int_gemm.cc \
 test/test_data.cc


 Testing using Bazel
 -------------------

 Alternatively, you can use Bazel to build and run tests. See the Bazel
 instruction in the above section on building. Once your Bazel workspace
 is set up, you can for instance do:

 $ bazel test gemmlowp:all


 Troubleshooting Compilation
 ===========================

 If you're having trouble finding the compiler, follow these instructions to
 build a standalone toolchain:
 https://developer.android.com/ndk/guides/standalone_toolchain.html

 Here's an example of setting up Clang 3.5:

 $ export INSTALL_DIR=~/toolchains/clang-21-stl-gnu
 $ $NDK/build/tools/make-standalone-toolchain.sh \
 --toolchain=arm-linux-androideabi-clang3.5 --platform=android-21 \
 --install-dir=$INSTALL_DIR
 $ export CXX="$INSTALL_DIR/bin/arm-linux-androideabi-g++ \
 --sysroot=$INSTALL_DIR/sysroot"

 Some compilers (e.g. the default clang++ in the same bin directory) don't
 support NEON assembly. The benchmark build process will issue a warning if
 support isn't detected, and you should make sure you're using a compiler like
 arm-linux-androideabi-g++ that does include NEON.


 Benchmarking
 ============

 The main benchmark is
   benchmark.cc
 It doesn't need to be linked to any
 other source file. We recommend building with assertions disabled (-DNDEBUG).

 For example, the benchmark can be built and run on an Android device by doing:

 $ ./scripts/test-android.sh test/benchmark.cc -DNDEBUG

 If GEMMLOWP_TEST_PROFILE is defined then the benchmark will be built with
 profiling instrumentation (which makes it slower) and will dump profiles.
 See next section on profiling.


 Profiling
 =========

 The profiling/ subdirectory offers a very simple non-interrupting sampling
 profiler that only requires pthreads (no signals).

 It relies on source code being instrumented with pseudo-stack labels.
 See profiling/instrumentation.h.
 A full example of using this profiler is given in profiling/profiler.h.


 Contributing
 ============

 Contribution-related discussion is always welcome on the gemmlowp
 mailing list (see above).

 We try to keep a current list of TODO items in the todo/ directory.
 Prospective contributors are welcome to pick one to work on, and
 communicate about it on the gemmlowp mailing list.

 Details of the contributing process, including legalese, are in CONTRIBUTING.

 Performance goals
 =================

 Our performance goals differ from typical GEMM performance goals in the
 following ways:

 1. We care not only about speed, but also about minimizing power usage.
    We specifically care about charge usage in mobile/embedded devices.
    This implies that we care doubly about minimizing memory bandwidth usage:
    we care about it, like any GEMM, because of the impact on speed, and we
    also care about it because it is a key factor of power usage.

 2. Most GEMMs are optimized primarily for large dense matrix sizes (>= 1000).
    We do care about large sizes, but we also care specifically about the
    typically smaller matrix sizes encountered in various mobile applications.
    This means that we have to optimize for all sizes, not just for large enough
    sizes.
	gemmlowp: a small self-contained low-precision GEMM library
	===========================================================

	This is not a full linear algebra library, only a GEMM library: it only does
	general matrix multiplication ("GEMM").

	The meaning of "low precision" is detailed in this document:
	doc/low-precision.txt

	Some of the general design is explained in
	doc/design.txt


	Disclaimer
	==========

	This is not an official Google product (experimental or otherwise), it is just
	code that happens to be owned by Google.


	Mailing list
	============

	gemmlowp-related discussion, about either development or usage, is welcome
	on this Google Group (mailing list / forum):

	https://groups.google.com/forum/#!forum/gemmlowp


	Portability, target platforms/architectures
	===========================================

	Should be portable to any platform with some C++11 and POSIX support,
	while we have optional optimized code paths for specific architectures.

	Required:
	C++11 (a small conservative subset of it)

	Required for some features:
	* Some POSIX interfaces:
	* pthreads (for multi-threaded operation and for profiling).
	* sysconf (for multi-threaded operation to detect number of cores;
	may be bypassed).

	Optional:
	Architecture-specific code paths use intrinsics or inline assembly.
	See "Architecture-specific optimized code paths" below.

	Architecture-specific optimized code paths
	==========================================

	We have some optimized code paths for specific instruction sets.
	Some are written in inline assembly, some are written in C++ using
	intrinsics. Both GCC and Clang are supported.

	At the moment, we have a full set of optimized code paths (kernels,
	packing and unpacking paths) only for ARM NEON, supporting both
	ARMv7 (32bit) and ARMv8 (64bit).

	We also have a partial set of optimized code paths (only kernels
	at the moment) for Intel SSE. It supports both x86 and x86-64 but
	only targets SSE4. The lack of packing/unpacking code paths means
	that performance isn't optimal yet.

	Details of what it takes to make an efficient port of gemmlowp, namely
	writing a suitable GEMM kernel and accompanying packing code, are
	explained in this file:
	doc/kernels.txt


	Public interfaces
	=================

	1. gemmlowp public interface
	----------------------------

	gemmlowp's main public interface is in the public/ subdirectory. The
	header to include is
	public/gemmlowp.h.
	This is a headers-only library, so there is nothing to link to.

	2. EightBitIntGemm standard interface
	-------------------------------------

	Additionally, the eight_bit_int_gemm/ subdirectory provides an
	implementation of the standard EightBitIntGemm interface. The header
	to include is
	eight_bit_int_gemm/eight_bit_int_gemm.h
	This is NOT a headers-only library, users need to link to
	eight_bit_int_gemm/eight_bit_int_gemm.cc.
	The API is similar to the standard BLAS GEMM interface, and implements
	C = A * B. If the transpose flags for a matrix argument are false, its memory
	order is treated as column major, and row major if its true.


	Building
	========

	Building by manually invoking your compiler
	-------------------------------------------

	Because gemmlowp is so simple, working with it involves only
	single-command-line compiler invokations. Therefore we expect that
	most people working with gemmlowp will either manually invoke their
	compiler, or write their own rules for their own preferred build
	system.

	Keep in mind (previous section) that gemmlowp itself is a pure-headers-only
	library so there is nothing to build, and the eight_bit_int_gemm library
	consists of a single eight_bit_int_gemm.cc file to build.

	For a Android gemmlowp development workflow, the scripts/ directory
	contains a script to build and run a program on an Android device:
	scripts/test-android.sh

	Building using Bazel
	--------------------

	That being said, we also maintain a Bazel BUILD system as part of
	gemmlowp. Its usage is not mandatory at all and is only one
	possible way that gemmlowp libraries and tests may be built. If
	you are interested, Bazel's home page is
	http://bazel.io/
	And you can get started with using Bazel to build gemmlowp targets
	by first creating an empty WORKSPACE file in a parent directory,
	for instance:

	$ cd gemmlowp/.. # change to parent directory containing gemmlowp/
	$ touch WORKSPACE # declare that to be our workspace root
	$ bazel build gemmlowp:all


	Testing
	=======

	Testing by manually building and running tests
	----------------------------------------------

	The test/ directory contains unit tests. The primary unit test is
	test/test.cc
	Since it covers also the EightBitIntGemm interface, it needs to be
	linked against
	eight_bit_int_gemm/eight_bit_int_gemm.cc
	It also uses realistic data captured from a neural network run in
	test/test_data.cc

	Thus you'll want to pass the following list of source files to your
	compiler/linker:
	test/test.cc
	eight_bit_int_gemm/eight_bit_int_gemm.cc
	test/test_data.cc

	The scripts/ directory contains a script to build and run a program
	on an Android device:
	scripts/test-android.sh

	It expects the CXX environment variable to point to an Android toolchain's
	C++ compiler, and expects source files (and optionally, cflags) as
	command-line parameters. To build and run the above-mentioned main unit test,
	first set CXX e.g.:

	$ export CXX=/some/toolchains/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-g++

	Then run:

	$ ./scripts/test-android.sh \
	test/test.cc \
	eight_bit_int_gemm/eight_bit_int_gemm.cc \
	test/test_data.cc


	Testing using Bazel
	-------------------

	Alternatively, you can use Bazel to build and run tests. See the Bazel
	instruction in the above section on building. Once your Bazel workspace
	is set up, you can for instance do:

	$ bazel test gemmlowp:all


	Troubleshooting Compilation
	===========================

	If you're having trouble finding the compiler, follow these instructions to
	build a standalone toolchain:
	https://developer.android.com/ndk/guides/standalone_toolchain.html

	Here's an example of setting up Clang 3.5:

	$ export INSTALL_DIR=~/toolchains/clang-21-stl-gnu
	$ $NDK/build/tools/make-standalone-toolchain.sh \
	--toolchain=arm-linux-androideabi-clang3.5 --platform=android-21 \
	--install-dir=$INSTALL_DIR
	$ export CXX="$INSTALL_DIR/bin/arm-linux-androideabi-g++ \
	--sysroot=$INSTALL_DIR/sysroot"

	Some compilers (e.g. the default clang++ in the same bin directory) don't
	support NEON assembly. The benchmark build process will issue a warning if
	support isn't detected, and you should make sure you're using a compiler like
	arm-linux-androideabi-g++ that does include NEON.


	Benchmarking
	============

	The main benchmark is
	benchmark.cc
	It doesn't need to be linked to any
	other source file. We recommend building with assertions disabled (-DNDEBUG).

	For example, the benchmark can be built and run on an Android device by doing:

	$ ./scripts/test-android.sh test/benchmark.cc -DNDEBUG

	If GEMMLOWP_TEST_PROFILE is defined then the benchmark will be built with
	profiling instrumentation (which makes it slower) and will dump profiles.
	See next section on profiling.


	Profiling
	=========

	The profiling/ subdirectory offers a very simple non-interrupting sampling
	profiler that only requires pthreads (no signals).

	It relies on source code being instrumented with pseudo-stack labels.
	See profiling/instrumentation.h.
	A full example of using this profiler is given in profiling/profiler.h.


	Contributing
	============

	Contribution-related discussion is always welcome on the gemmlowp
	mailing list (see above).

	We try to keep a current list of TODO items in the todo/ directory.
	Prospective contributors are welcome to pick one to work on, and
	communicate about it on the gemmlowp mailing list.

	Details of the contributing process, including legalese, are in CONTRIBUTING.

	Performance goals
	=================

	Our performance goals differ from typical GEMM performance goals in the
	following ways:

	1. We care not only about speed, but also about minimizing power usage.
	We specifically care about charge usage in mobile/embedded devices.
	This implies that we care doubly about minimizing memory bandwidth usage:
	we care about it, like any GEMM, because of the impact on speed, and we
	also care about it because it is a key factor of power usage.

	2. Most GEMMs are optimized primarily for large dense matrix sizes (>= 1000).
	We do care about large sizes, but we also care specifically about the
	typically smaller matrix sizes encountered in various mobile applications.
	This means that we have to optimize for all sizes, not just for large enough
	sizes.