commit	06502e2bf0b77f97ed0cfd63e41209df56e13733	[log] [tgz]
author	David Neto <dneto@google.com>	Wed Sep 21 12:36:09 2016 -0400
committer	David Neto <dneto@google.com>	Wed Sep 21 13:18:58 2016 -0400
tree	a62347cf0e7b4e4e6d6cf7a94dbde158555a37ec
parent	c0640e0789c9c9daea61fac38c6535abd0744fa2 [diff]
parent	f18e1f293b7cf8f24a42587b48f35bc49912a5dd [diff]

Merge remote-tracking branch 'aosp/upstream-master' into shaderc-update-master

Pulls in the following:
f18e1f2 Add a C++ interface to the optimization functionalities.
1bc0b27 Allow changing MessageConsumer in Pass & PassManager.
f8dbed0 Change BuildModule() to accept pointer-size pair for binary.
2cbb2cc Change interface of Pass::Process() to return possible failures.
12b5777 Provide friendly names for shader ballot BuiltIns
bf467da Update validate interface to accept binary pointers
3590279 Set the default source name as "input".
6519fce Start v2016.6-dev
f9763ac Finalize v2016.5
9382035 Add basic support for SPV_KHR_shader_ballot
8654caa Prepare the C++ interface for publication.
b54686d Move functions for building modules outside of the C++ interface.
869440e Let SPIRV_ASSERT() exit the program if the assertion fails.
114af21 Add MessageConsumer to PassManager, Pass, and analysis interfaces.
bfd7cd6 Add utility macros for logging messages.
755f97f Add a callback mechanism for communicating messages to callers.
e0dd033 Make friendly names for built-in variables.
247e024 Update CHANGES to say #414 is fixed
1f3fb50 Fix validator: OpUndef can be member of a constant composite
4e2ed04 Clarify: Ok to pass null diagnostic to spvBinaryParse
5c9080e Fix validator SSA check: Phi can use its own value sometimes
66f5b4b Build "spec id->default val str" mapping from string
5ac6352 Guard against nullptr in number parsing utilities
abf5793 Set default values for spec const decorated by SpecId
26b51ef Update CMake condition for Emacs load/save helper
6accefc add spirv-dis/spirv-as support for emacs when loading .spv binaries
049f80f Add IsSigned() to opt::Analysis::Integer.
1773b95 Pull out the number parsing logic
a35919c Merge diagnostic tests and single them out in a separate binary.
219f0cc Create separate binaries for validation tests.
ef600df Prefix test binaries with `test_`, rename UnitSPIRV, update README.
860030f Travis CI: mark build as finished once a job failed.
9222467 Kill the spvCheck macro which is just alias to if-statement.
53c8433 Update README about the optimization library and tool.
dcd7fed Say contributors have to click through the CLA
881816f Add test for presence of copyrights and license terms
e70a957 Start v2016.5-dev
9fc8658 Relicense SPIRV-Tools under Apache 2.0
6f35405 Add check_copyright to check or update copyright.
0e001b7 Fix Google copyright lines
5abf408 Kill the spvCheckReturn macro.
75bf78c Kill diagnostic_helper & the wrapping DIAGNOSTIC macro.
2ad3b74 Tests for preserving line debug insts even killing its affecting targets.
eb60e29 Unify constants pass
909d7f9 Refactor CapabilitySet into templated class EnumSet
2ce6725 Add forwarding so that passes' ctor can have args.
273920c Handle capabilities larger than value 63
47f2a14 Add libspirv::CapabilitySet
7698270 Get annotation instructions of a given id.
ecb5692 Add AppendNames() interface for AssemblyBuilder
d9e63de Create a separate header and source file for each pass.
380f36e Fold spec constants to normal constants (values fixed)
1d59aa0 Pass manager recomputes Id bound automatically.
b1b2cac Create const overloads for begin() and end() in Instruction.
923a459 Fix typo in spirv-opt help
d7b6362 Add Module::ComputeIdBound
0a175eb Add tests for decoration_empty().
ec07a07 Add a method in the types to check whether there is no decoration on a type.
58e1a1e Test ir::Instruction::GetOperand and GetInOperand
cc60cab IrLoader gracefully handles incomplete blocks and functions
53013cc Add Instruction begin and end mutable iterators
2607bbf Add Instruction cbegin and cend to access operands
97fc6aa ForEachInst optionally runs on attached debug line insts
e96a881 Travis CI: add build configuration for Android NDK with cmake.
0d4b73c Start v2016.4-dev
a7a84bd Finalize v2016.3
3c35b63 Add spirv-lesspipe.sh
97d06a3 Mere mention of PointSize BuiltIn does not require Shader capability
f51d823 Add a home brewed make_unique to ease creating unique_ptrs.
6f13c73 Remove some unused macros and functions.
f33aacd CHANGES captures fix for issue 365
358cb29 Relax ClipDistance, CullDistance capability check in all environments
ccabcc4 Add opt::ir::Module::SetIdBound
7e775aa Get the number of elements for Vector and Matrix type
b891845 Refine the DefUseManager
6c5bdc9 Get the width from integer and float type

Test: ndk/checkbuild.py on Linux; manually ran unit tests on Windows
Change-Id: Ia552c511929fb196d33efec54c0d39d023e0c263

tree: a62347cf0e7b4e4e6d6cf7a94dbde158555a37ec

README.md

SPIR-V Tools

Overview

The SPIR-V Tools project provides an API and commands for processing SPIR-V modules.

The project includes an assembler, binary module parser, disassembler, validator, and optimizer for SPIR-V. Except for the optimizer, all are based on a common static library. The library contains all of the implementation details, and is used in the standalone tools whilst also enabling integration into other code bases directly. The optimizer implementation resides in its own library.

The interfaces are still under development, and are expected to change.

SPIR-V is defined by the Khronos Group Inc. See the SPIR-V Registry for the SPIR-V specification, headers, and XML registry.

Verisoning SPIRV-Tools

See CHANGES for a high level summary of recent changes, by version.

SPIRV-Tools project version numbers are of the form vyear.index and with an optional -dev suffix to indicate work in progress. For exampe, the following versions are ordered from oldest to newest:

v2016.0
v2016.1-dev
v2016.1
v2016.2-dev
v2016.2

Use the --version option on each command line tool to see the software version. An API call reports the software version as a C-style string.

Supported features

Assembler, binary parser, and disassembler

Based on SPIR-V version 1.1 Rev 3
Support for extended instruction sets:
- GLSL std450 version 1.0 Rev 3
- OpenCL version 1.0 Rev 2
Support for SPIR-V 1.0 (with or without additional restrictions from Vulkan 1.0)
Assembler only does basic syntax checking. No cross validation of IDs or types is performed, except to check literal arguments to OpConstant, OpSpecConstant, and OpSwitch.

See syntax.md for the assembly language syntax.

Validator

Warning: The validator is incomplete.

Optimizer

Warning: The optimizer is still under development and its library interface is not yet published.

Currently supported optimizations:

For the latest list, please refer to spirv-opt --help.

Source code

The SPIR-V Tools are maintained by members of the The Khronos Group Inc., at https://github.com/KhronosGroup/SPIRV-Tools.

Contributions via merge request are welcome. Changes should:

Be provided under the Apache 2.0. You‘ll be prompted with a one-time “click-through” Contributor’s License Agreement (CLA) dialog as part of submitting your pull request or other contribution to GitHub.
Include tests to cover updated functionality.
C++ code should follow the Google C++ Style Guide.
Code should be formatted with clang-format. Settings are defined by the included .clang-format file.

We intend to maintain a linear history on the GitHub master branch.

Source code organization

external/googletest: Intended location for the googletest sources, not provided
include/: API clients should add this directory to the include search path
external/spirv-headers: Intended location for SPIR-V headers, not provided
include/spirv-tools/libspirv.h: C API public interface
source/: API implementation
test/: Tests, using the googletest framework
tools/: Command line executables

Tests

The project contains a number of tests, used to drive development and ensure correctness. The tests are written using the googletest framework. The googletest source is not provided with this project. There are two ways to enable tests:

If SPIR-V Tools is configured as part of an enclosing project, then the enclosing project should configure googletest before configuring SPIR-V Tools.
If SPIR-V Tools is configured as a standalone project, then download the googletest source into the <spirv-dir>/external/googletest directory before configuring and building the project.

Note: You must use a version of googletest that includes a fix for googletest issue 610. The fix is included on the googletest master branch any time after 2015-11-10. In particular, googletest must be newer than version 1.7.0.

Build

The project uses CMake to generate platform-specific build configurations. Assume that <spirv-dir> is the root directory of the checked out code:

cd <spirv-dir>
git clone https://github.com/KhronosGroup/SPIRV-Headers.git external/spirv-headers
git clone https://github.com/google/googletest.git external/googletest # optional

mkdir build && cd build
cmake [-G <platform-generator>] <spirv-dir>

Once the build files have been generated, build using your preferred development environment.

CMake options

The following CMake options are supported:

SPIRV_COLOR_TERMINAL={ON|OFF}, default ON - Enables color console output.
SPIRV_SKIP_TESTS={ON|OFF}, default OFF- Build only the library and the command line tools. This will prevent the tests from being built.
SPIRV_SKIP_EXECUTABLES={ON|OFF}, default OFF- Build only the library, not the command line tools and tests.
SPIRV_USE_SANITIZER=<sanitizer>, default is no sanitizing - On UNIX platforms with an appropriate version of clang this option enables the use of the sanitizers documented here. This should only be used with a debug build.
SPIRV_WARN_EVERYTHING={ON|OFF}, default OFF - On UNIX platforms enable more strict warnings. The code might not compile with this option enabled. For Clang, enables -Weverything. For GCC, enables -Wpedantic. See CMakeLists.txt for details.
SPIRV_WERROR={ON|OFF}, default ON - Forces a compilation error on any warnings encountered by enabling the compiler-specific compiler front-end option.

Library

Usage

The library provides a C API, but the internals use C++11.

In order to use the library from an application, the include path should point to <spirv-dir>/include, which will enable the application to include the header <spirv-dir>/include/spirv-tools/libspirv.h then linking against the static library in <spirv-build-dir>/source/libSPIRV-Tools.a or <spirv-build-dir>/source/SPIRV-Tools.lib.

SPIRV-Tools CMake target: Creates the static library:
- <spirv-build-dir>/source/libSPIRV-Tools.a on Linux and OS X.
- <spirv-build-dir>/source/libSPIRV-Tools.lib on Windows.

Entry points

The interfaces are still under development, and are expected to change.

There are three main entry points into the library.

spvTextToBinary: An assembler, translating text to a binary SPIR-V module.
spvBinaryToText: A disassembler, translating a binary SPIR-V module to text.
spvBinaryParse: The entry point to a binary parser API. It issues callbacks for the header and each parsed instruction. The disassembler is implemented as a client of spvBinaryParse.
spvValidate implements the validator functionality. Incomplete

Command line tools

Command line tools, which wrap the above library functions, are provided to assemble or disassemble shader files. It's a convention to name SPIR-V assembly and binary files with suffix .spvasm and .spv, respectively.

Assembler tool

The assembler reads the assembly language text, and emits the binary form.

The standalone assembler is the exectuable called spirv-as, and is located in <spirv-build-dir>/tools/spirv-as. The functionality of the assembler is implemented by the spvTextToBinary library function.

spirv-as - the standalone assembler
- <spirv-dir>/tools/as

Use option -h to print help.

Disassembler tool

The disassembler reads the binary form, and emits assembly language text.

The standalone disassembler is the executable called spirv-dis, and is located in <spirv-build-dir>/tools/spirv-dis. The functionality of the disassembler is implemented by the spvBinaryToText library function.

spirv-dis - the standalone disassembler
- <spirv-dir>/tools/dis

Use option -h to print help.

The output includes syntax colouring when printing to the standard output stream, on Linux, Windows, and OS X.

Optimizer tool

The optimizer processes a SPIR-V binary module, applying transformations in the specified order.

This is a work in progress, with initially only few available transformations.

spirv-opt - the standalone optimizer
- <spirv-dir>/tools/opt

Validator tool

Warning: This functionality is under development, and is incomplete.

The standalone validator is the executable called spirv-val, and is located in <spirv-build-dir>/tools/spirv-val. The functionality of the validator is implemented by the spvValidate library function.

The validator operates on the binary form.

spirv-val - the standalone validator
- <spirv-dir>/tools/val

Control flow dumper tool

The control flow dumper prints the control flow graph for a SPIR-V module as a GraphViz graph.

This is experimental.

spirv-cfg - the control flow graph dumper
- <spirv-dir>/tools/cfg

Utility filters

spirv-lesspipe.sh - Automatically disassembles .spv binary files for the less program, on compatible systems. For example, set the LESSOPEN environment variable as follows, assuming both spirv-lesspipe.sh and spirv-dis are on your executable search path:
```
 export LESSOPEN='| spirv-lesspipe.sh "%s"'
```
Then you page through a disassembled module as follows:
```
less foo.spv
```
- The spirv-lesspipe.sh script will pass through any extra arguments to spirv-dis. So, for example, you can turn off colours and friendly ID naming as follows:
```
export LESSOPEN='| spirv-lesspipe.sh "%s" --no-color --raw-id'
```
50spirv-tools.el - Automatically disassembles ‘.spv’ binary files when loaded into the emacs text editor, and re-assembles them when saved, provided any modifications to the file are valid. This functionality must be explicitly requested by defining the symbol SPIRV_TOOLS_INSTALL_EMACS_HELPERS as follows:
```
cmake -DSPIRV_TOOLS_INSTALL_EMACS_HELPERS=true ...
```
In addition, this helper is only installed if the directory /etc/emacs/site-start.d exists, which is typically true if emacs is installed on the system.
Note that symbol IDs are not currently preserved through a load/edit/save operation. This may change if the ability is added to spirv-as.

Tests

Tests are only built when googletest is found. Use ctest to run all the tests.

Future Work

Assembler and disassembler

The disassembler could emit helpful annotations in comments. For example:
- Use variable name information from debug instructions to annotate key operations on variables.
- Show control flow information by annotating OpLabel instructions with that basic block's predecessors.
Error messages could be improved.

Validator

This is a work in progress.

Licence

Full license terms are in LICENSE

Copyright (c) 2015-2016 The Khronos Group Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.