tag	af1a58205589d604e03ee6bd544aed1f894e7b0c
tagger	The Android Open Source Project <initial-contribution@android.com>	Mon Jul 25 15:49:31 2016 -0700
object	9fc206608c21434e765daaca7c52410f60263bfa

NDK Release 13 Beta 1 (3057941)

commit	9fc206608c21434e765daaca7c52410f60263bfa	[log] [tgz]
author	David Neto <dneto@google.com>	Mon Jun 20 10:37:38 2016 -0400
committer	David Neto <dneto@google.com>	Mon Jun 20 10:37:38 2016 -0400
tree	3c020d6f32aa995352705845c3aa267ee1d6db6f
parent	c1ccd4bf91d665a19c7ed2694fe65bf1738a6845 [diff]
parent	19b92fff7ec0f5973d45434764f79e8f13c186f9 [diff]

Merge remote-tracking branch 'aosp/upstream-master' into update-shaderc-sources

Add the following, including ec1476b std::map allocator const-key fix for
newer libc++:

19b92ff HLSL: Basic array grammar.
93a162a HLSL: Handle "." for structure dereference and swizzle.
5aa59e2 HLSL: Map parameter qualifiers from generic to function-specific and entry-point specific.
c3387d3 HLSL: Support semantics in function parameter declarations.
21f1286 Front-end: Implement GL_EXT_shader_non_constant_global_initializers.
b901ade SPV: Non-functional: Condense SPV-related versioning, and rationalize all uses.
6533648 SPV: projective texturing: move the project coordinate.
76d4dfc SPV non-functional: minor readability improvements for texturing.
ac666e7 Front-end ESSL: require precision declarations for images.
9000bd5 Front-end: Make ES floatBitsToInt/intBitsToFloat/etc. all be highp.
c51287d SPV: Update to spec. decisions (and issue #205) for barrier().
01de90b Merge pull request #339 from steve-lunarg/intrinsics
7b04bdd Merge pull request #340 from dneto0/const-key-in-std-maps
ec1476b Tracking allocator must use const key
5891070 Implement atomic ops, bit conversions, fix fwidth stage mask, fix saturate dest modifier.
cd784bc Merge pull request #337 from steve-lunarg/intrinsics
af3674d Merge pull request #338 from Nekotekina/fix1
5dd29c8 Compilation fix (mingw64)
630dd7d HLSL: Flesh out misc. declaration grammar: semantics/registers/annotations/precise/etc.
e6e7494 HLSL: Implement basic "struct" grammar.
592860c Add decompositions for some HLSL intrinsics.
41ebc42 Merge pull request #335 from jekstrand/sampler-params
077e052 HLSL: Implement proper nesting of symbol-table scopes and identifier searching.
71351de HLSL: Add all int/float/bool/uint matrix types, void for functions, and a few others.
ed15ef1 GlslangToSpv: Pass the pointer directly into the client function for opaque types
228546a Types: Add an isOpaque() helper
c24cc29 GlslangToSpv: Properly handle opaque types in TranslateStorageClass
5022d68 Merge pull request #333 from dneto0/defer-capabilities-for-some-struct-members-until-used
a901ffe Defer capability decl for ClipDistance, CullDistance, PointSize until actual use
95d85ce Merge pull request #331 from jherico/jherico-patch-1
880bf36 Fix Ubuntu 16.10 build
71d55c0 Merge pull request #128 from dekimir/clang-format
80cb324 Front-end: Fix issue #146: which versions allow double in/out.
738314f Revert SpvBuilder.* to master versions.
e592652 Increase column limit to 120.
f377f52 Merge branch 'master' of github.com:KhronosGroup/glslang into clang-format
133253b Front-end: Fix issue #147: ensure layout(index=N) has N in [0,1].
119f8f6 HLSL: Flesh out the loop grammar and productions.
0d2b6de HLSL: Attribute grammar and if-else grammar/productions.
21472ae HLSL: Finish skeletan of the "statement" grammar.
1cc1a28 HLSL: 1) Implement lookahead buffers/stacks for token advance/recede, 2) use it for cast operation.
080cdc1 Merge pull request #328 from steve-lunarg/intrinsics
643fcb5 Add negative intrinsics tests
54e3bb8 Merge pull request #327 from steve-lunarg/empty-statement-fix
ef764a2 Fix for empty statement segfault.
9f1f818 Merge pull request #326 from greg-lunarg/greg-mr4
438bb60 Merge pull request #322 from steve-lunarg/intrinsics
39d5e71 remapper: do not eliminate interface variables in dce by default
4b67732 Initial implementation of direct-mapped subset of HLSL intrinsics with type subset.
85b52f3 Merge pull request #323 from dneto0/mingw-static-link-against-basic-runtimes
a5c33d6 SPV: Fix issue #320: Fetch needs to extract the image.
b37dc0e For MinGW, statically link exes against basic runtimes
98f2b3b Merge pull request #311 from jherico/debug_postfix
85b3326 Merge pull request #317 from iostrowsINTEL/double_matrix_fix
af7f1c8 Fix double matrix creation
548c3ad Front-end: Complete GL_ARB_compute_shader, previous commit was missing new barriers.
d94c003 Front-end: Complete GL_ARB_compute_shader implementation.
0f5e3ad Fix issue #313: Catch internal attempts to modify built-in symbols that don't exist.
5a7f0ef Merge pull request #230 from dekimir/incresconst
7d81419 Merge pull request #309 from jekstrand/sampler-params
747c328 Merge pull request #308 from thp/multiple-es-compilation-units
76d0ac1 SPV: Don't shadow sampler parameters when performing function calls
1e194e8 Produce different names for files in debug builds on MSVC
7bfd08d Check for linking multiple ES shaders to the same stage
0bb546f Fix test case for the "empty" linker test case
b40a6d6 Use version/profile from first compilation stage
63d4794 Add test case: Link 2 ESSL fragment shader units
0c968f9 Merge pull request #306 from johnkslang/finish-spec-const-semantics
d82c906 Vulkan: Finish semantics for what creates spec-const-semantics.
479b4c7 Merge pull request #305 from steve-lunarg/intrinsics
0ae28ea Add base class TParseables for intrinsic / builtin generation. Add stubbed HLSL derivation.  GLSL derivation is still called TBuiltIns, for historical compatibility.
87a94fc Merge pull request #302 from amdrexu/bugfix
bbceed7 SPV: Fix an issue of interpolation decoration.
2921e0c KHR_vulkan_glsl: name mangle distinguish pure textures.
22e0d41 SPV: Don't put locations on OpTypeStruct. Related to Issue #291.
75ce845 Merge pull request #301 from dneto0/fix-android-build-atoi-in-cstddef
2ed1d9b atoi comes from stddef.h or cstddef
eee0c73 Merge pull request #297 from antiagainst/config-tests
393d74f Merge pull request #300 from Qining/fix-slow-down-in-no-contraction-propagation
507308b remove redundant 'return false'
0133c12 HLSL: Add more matrix types to the grammar.
8d72f1a Full stack: distinguish between a scalar and a vector of size 1.
e5712a2 Fix the slow down in noContraction propagation
823fc65 SPV: Don't emit memory barrier for ESSL barrier(), but still do for GLSL barrier().
eb6d950 Merge pull request #298 from thp/remove-unused-prototype
00852b1 Remove declaration for ShGetPhysicalAttributeBindings
1b14172 Test compiling shaders with given resource limits using GTest.
8a9b1ee Rename DefaultResourceLimits.* to ResourceLimits.*.
b5cfd79 Merge pull request #288 from thp/attribute-reflection
bef7428 Update test cases for vertex attribute reflection
d6e1a5b Add support for querying vertex attributes in reflection API
9349108 Merge pull request #295 from amdrexu/bugfix
e3e92d3 Merge pull request #293 from antiagainst/link-tests
3d9165f Parser: Redeclaration of gl_CullDistance is disallowed mistakenly.
2f1ee45 Run link tests in the GTest framework.
f62b519 Merge pull request #294 from scyganINTEL/scygan-fix-io_storage_structs-location
2c86427 Fix missing location decoration for structures put directly on input/output interfaces
fc697cc Fix several comments in GTest fixture.
9af54c3 Non-functional: Fix some comments English and brace formatting in recent merges.
5b3f755 Merge branch 'scyganINTEL-struct_member_decorations'
6e0985d Merge branch 'struct_member_decorations' of https://github.com/scyganINTEL/glslang into scyganINTEL-struct_member_decorations
44b30cb Merge pull request #286 from amdrexu/bugfix
c1992e5 SPV: Correct an issue in createUnaryMatrixOperation().
bedde87 Merge pull request #285 from baldurk/vs2010-fixes
ebb5053 SPV: Don't add clip/cull distance capabilities unless used.
e5f2939 Tests: Add a set of gtest-file-based HLSL tests.
6b32ae1 Compile fix - if _MSC_VER is undefined, _MSC_VER < 1700 is true!
a8018b8 [VS2010] Define strtoll() and atoll() functions
bf2c88b [VS2010] Locally define std::to_string on VS2010 as well as android
ab44ba7 [VS2010] Remove use of ranged-for loops
c1d81cb [VS2010] Remove use of "= default" and "= delete"
786cf4d [VS2010] Change "using X = Y;" to "typedef Y X;"
e0a2477 Merge pull request #284 from antiagainst/gtest-hlsl
7a27de6 Merge pull request #253 from amdrexu/bugfix2
7fde8bc Merge pull request #281 from amdrexu/bugfix
b9e9743 Merge pull request #282 from baldurk/android-warning-fix
d6f0ed2 Add support for testing file-based HLSL source code in GTest.
f2d75f7 Use DetachThreadLinux wrapper on android
62dda78 Parser: Add missing vector relational functions for double type.
73e3ce7 SPV: Handle matrix's OpFConvert vector by vector.
4678ca9 HLSL: Add function call syntax and AST building.
9db3117 Merge pull request #278 from greg-lunarg/greg-mr3
36962c7 Merge pull request #276 from amdrexu/bugfix
9f0cbdc Merge pull request #277 from amdrexu/bugfix3
2713ed7 fix for -Werror=non-virtual-dtor for Android system builds
5839031 SPV: Add capability ImageGatherExtended if necessary.
827b23b Parser: Redeclare gl_ClipDistance/gl_CullDistance should update their array sizes.
fc6ac78 Merge pull request #274 from antiagainst/stoul-strtoul
20daa7f Use strtoul instead of stoul for Android.
b2bd53b Merge pull request #273 from antiagainst/appveyor-readme-status
0e101c9 Add badge status for Appveyor build in README.
210c6bf Merge pull request #272 from antiagainst/appveyor
cb3236d Use std::string for TBD/missing functionality reporting.
f36d6e3 Add test configuration for Appveyor.
3357d87 Merge pull request #222 from Qining/support-precise
7b3096a Merge pull request #270 from scyganINTEL/bitfield_buildins
dc4c9a7 Merge pull request #271 from greg-lunarg/greg-mr2
e543cc3 remove double underscore
bb0ac54 Rebase to d3d3ce7160001846ede74978954e67f4f9fd84e4,update comments, rename a member variable name, fix format issue
5bdf49c fix check for non-positive array size
9c56d84 Use proper precision qualifiers for bitfield build-in functions on ES
5bec2b5 integer type operation should not be noContraction
34d2571 simplified the symbol definition collector code, add assert()
0c96db5 add a test for matrix, fix a variable name
25262b3 Resolve comments
015150e Removed the redundant functionalities in the assignee checking traverser
1a0d93f Fix the test of precise output function parameter
9220dbb Precise and noContraction propagation
d3d3ce7 Comments only.
8add151 SPIR-V do not decorate: members of plain structs with location; non-top level members with interp.
3bdf242 README: Editorial update.
91cef52 SPV: Non-functional: modularize createInvocationsOperation() from the previous commit.
213d4be Merge branch 'amdrexu-feature2'
baa9fa5 Merge branch 'feature2' of https://github.com/amdrexu/glslang into amdrexu-feature2
e2d06db Merge pull request #268 from AWoloszyn/update-cmake
5cc344d Front-End: precise: capture noContraction on output parameter declarations.
076879d Merge pull request #266 from dekimir/kfixstoll
0b887d0 Install the test executable alongside glslangValidator
db0eaf9 Updated cmake to better organize folders and options.
0a0db9e Merge branch 'amdrexu-feature'
c8a5676 Merge branch 'feature' of https://github.com/amdrexu/glslang into amdrexu-feature
a25352e Use strtoll, as stoll is unavailable on Android.
6dea76a Merge pull request #257 from antiagainst/ostrstream
0542ac8 Merge pull request #261 from antiagainst/make-tests-clear
d7b552a Improve README steps for building and testing glslang.
f5e33b0 Explain organization and build steps for source code and tests.
17535f7 Create a new logger class for all messages w.r.t. SPIR-V build.
09caf12 Avoid printing to stdout directly in library functions.
338b185 Implement the extension GL_ARB_shader_group_vote.
574ab04 Implement the extension GL_ARB_shader_ballot
97f4e0f Merge pull request #263 from antiagainst/cmake-minimum-required
3f46053 Remove duplicated cmake_minimum_required() calls.
3c1e080 Front-end precise: Propagate noContraction up the AST when dereferencing structs.
17f0786 Parser: Precise: Recognize 'precise', tag types, and do related semantic checking.
758bb55 Bison: Non-functional: Move to revision 3.0.4.
952c2a0 Merge pull request #262 from antiagainst/remove-obselete-src
d145f85 Remove the obselete and incorrect LinusDoAll.bash.
a655569 Merge pull request #260 from antiagainst/cmake-version
045e02a Bump minimum required CMake version to 2.8.12.
52e1db8 Merge pull request #258 from antiagainst/travis
c4ebb55 Add configuration for Travis to test on Linux and Mac OS X.
34fb036 HLSL: Add (almost) full expression grammar: Binary, unary (pre/post-fix), assign, ...
9c86c6a HLSL: Separate out token stream handling from grammar recognition.
e512cd9 Vulkan: Add the #define VULKAN 100 when compiled for Vulkan.
af45921 Correct precision qualification on built-in functions.
f88a5c7 Merge pull request #256 from DavidYen/missing_license
f67f912 Removed unused files SetupLinux.sh, index.php.
19f3f47 Merge pull request #248 from amdrexu/feature2
8ff43de Implement the extension GL_ARB_gpu_shader_int64
0042244 Added missing headers to SetupLinux.sh and index.php.
010e93f Merge pull request #228 from Qining/fix-infinite-loop-due-to-eof-missing
94a89d5 add .err file for eof_missing test
19647a3 Fix the infinite loop when an input file lacks EOF
ba00f67 Merge pull request #251 from amdrexu/bugfix
c7d3656 SPV: Use OpLogicalEqual/OpLogicalNotEqual for boolean type comparison.
d6abcee Merge pull request #247 from dneto0/no-mutex
10223e8 Remove use of std::mutex in gtest code.
63b280b Merge pull request #241 from Qining/fix-spec-const-construct-matrix
1f2820a fix the problem that spec constant composite instruction being used when only front-end constants are used in the constructor
27e04a0 Fix spec const construtor for matrix and vector
51eb91f Make two more IncludeResult members const.
5797975 Merge branch 'master' of github.com:KhronosGroup/glslang into clang-format
24c6bb3 Put braces on new line after if/for/switch/case.
8a4e22c Fully restore copyright spacing.
37c2a2d Merge branch 'master' of github.com:KhronosGroup/glslang into clang-format
757b3d3 Don't reflow comments.
778b876 Add .clang-format.

tree: 3c020d6f32aa995352705845c3aa267ee1d6db6f

README.md

Also see the Khronos landing page for glslang as a reference front end:

https://www.khronos.org/opengles/sdk/tools/Reference-Compiler/

The above page includes where to get binaries, and is kept up to date regarding the feature level of glslang.

glslang

An OpenGL and OpenGL ES shader front end and validator.

There are two components:

A front-end library for programmatic parsing of GLSL/ESSL into an AST.
A standalone wrapper, glslangValidator, that can be used as a shader validation tool.

How to add a feature protected by a version/extension/stage/profile: See the comment in glslang/MachineIndependent/Versions.cpp.

Things left to do: See Todo.txt

Execution of Standalone Wrapper

To use the standalone binary form, execute glslangValidator, and it will print a usage statement. Basic operation is to give it a file containing a shader, and it will print out warnings/errors and optionally an AST.

The applied stage-specific rules are based on the file extension:

.vert for a vertex shader
.tesc for a tessellation control shader
.tese for a tessellation evaluation shader
.geom for a geometry shader
.frag for a fragment shader
.comp for a compute shader

There is also a non-shader extension

.conf for a configuration file of limits, see usage statement for example

Building

Dependencies

CMake: for generating compilation targets.
bison: optional, but needed when changing the grammar (glslang.y).
googletest: optional, but should use if making any changes to glslang.

Build steps

1) Check-Out External Projects

cd <the directory glslang was cloned to, External will be a subdirectory>
git clone https://github.com/google/googletest.git External/googletest

2) Configure

Assume the source directory is $SOURCE_DIR and the build directory is $BUILD_DIR:

For building on Linux (assuming using the Ninja generator):

cd $BUILD_DIR

cmake -GNinja -DCMAKE_BUILD_TYPE={Debug|Release|RelWithDebInfo} \
      -DCMAKE_INSTALL_PREFIX=`pwd`/install $SOURCE_DIR

For building on Windows:

cmake $SOURCE_DIR -DCMAKE_INSTALL_PREFIX=`pwd`/install
# The CMAKE_INSTALL_PREFIX part is for testing (explained later).

The CMake GUI also works for Windows (version 3.4.1 tested).

3) Build and Install

# for Linux:
ninja install

# for Windows:
cmake --build . --config {Release|Debug|MinSizeRel|RelWithDebInfo} \
      --target install

If using MSVC, after running CMake to configure, use the Configuration Manager to check the INSTALL project.

If you need to change the GLSL grammar

The grammar in glslang/MachineIndependent/glslang.y has to be recompiled with bison if it changes, the output files are committed to the repo to avoid every developer needing to have bison configured to compile the project when grammar changes are quite infrequent. For windows you can get binaries from GnuWin32.

The command to rebuild is:

bison --defines=MachineIndependent/glslang_tab.cpp.h
      -t MachineIndependent/glslang.y
      -o MachineIndependent/glslang_tab.cpp

The above command is also available in the bash script at glslang/updateGrammar.

Testing

Right now, there are two test harnesses existing in glslang: one is Google Test, one is the runtests script. The former runs unit tests and single-shader single-threaded integration tests, while the latter runs multiple-shader linking tests and multi-threaded tests.

Running tests

The runtests script requires compiled binaries to be installed into $BUILD_DIR/install. Please make sure you have supplied the correct configuration to CMake (using -DCMAKE_INSTALL_PREFIX) when building; otherwise, you may want to modify the path in the runtests script.

Running Google Test-backed tests:

cd $BUILD_DIR

# for Linux:
ctest

# for Windows:
ctest -C {Debug|Release|RelWithDebInfo|MinSizeRel}

# or, run the test binary directly
# (which gives more fine-grained control like filtering):
<dir-to-glslangtests-in-build-dir>/glslangtests

Running runtests script-backed tests:

cd $SOURCE_DIR/Test && ./runtests

Contributing tests

Test results should always be included with a pull request that modifies functionality.

If you are writing unit tests, please use the Google Test framework and place the tests under the gtests/ directory.

Integration tests are placed in the Test/ directory. It contains test input and a subdirectory baseResults/ that contains the expected results of the tests. Both the tests and baseResults/ are under source-code control.

Google Test runs those integration tests by reading the test input, compiling them, and then compare against the expected results in baseResults/. The integration tests to run via Google Test is registered in various gtests/*.FromFile.cpp source files. glslangtests provides a command-line option --update-mode, which, if supplied, will overwrite the golden files under the baseResults/ directory with real output from that invocation. For more information, please check gtests/ directory's README.

For the runtests script, it will generate current results in the localResults/ directory and diff them against the baseResults/. The integration tests to run via the runtests script is registered via various Test/test-* text files and Test/testlist. When you want to update the tracked test results, they need to be copied from localResults/ to baseResults/. This can be done by the bump shell script.

The list of files tested comes from testlist, and lists input shaders in this directory, which must all be public for this to work. However, you can add your own private list of tests, not tracked here, by using localtestlist to list non-tracked tests. This is automatically read by runtests and included in the diff and bump process.

Programmatic Interfaces

Another piece of software can programmatically translate shaders to an AST using one of two different interfaces:

A new C++ class-oriented interface, or
The original C functional interface

The main() in StandAlone/StandAlone.cpp shows examples using both styles.

C++ Class Interface (new, preferred)

This interface is in roughly the last 1/3 of ShaderLang.h. It is in the glslang namespace and contains the following.

const char* GetEsslVersionString();
const char* GetGlslVersionString();
bool InitializeProcess();
void FinalizeProcess();

class TShader
    bool parse(...);
    void setStrings(...);
    const char* getInfoLog();

class TProgram
    void addShader(...);
    bool link(...);
    const char* getInfoLog();
    Reflection queries

See ShaderLang.h and the usage of it in StandAlone/StandAlone.cpp for more details.

C Functional Interface (orignal)

This interface is in roughly the first 2/3 of ShaderLang.h, and referred to as the Sh*() interface, as all the entry points start Sh.

The Sh*() interface takes a “compiler” call-back object, which it calls after building call back that is passed the AST and can then execute a backend on it.

The following is a simplified resulting run-time call stack:

ShCompile(shader, compiler) -> compiler(AST) -> <back end>

In practice, ShCompile() takes shader strings, default version, and warning/error and other options for controlling compilation.

Basic Internal Operation

Initial lexical analysis is done by the preprocessor in MachineIndependent/Preprocessor, and then refined by a GLSL scanner in MachineIndependent/Scan.cpp. There is currently no use of flex.
Code is parsed using bison on MachineIndependent/glslang.y with the aid of a symbol table and an AST. The symbol table is not passed on to the back-end; the intermediate representation stands on its own. The tree is built by the grammar productions, many of which are offloaded into ParseHelper.cpp, and by Intermediate.cpp.
The intermediate representation is very high-level, and represented as an in-memory tree. This serves to lose no information from the original program, and to have efficient transfer of the result from parsing to the back-end. In the AST, constants are propogated and folded, and a very small amount of dead code is eliminated.
To aid linking and reflection, the last top-level branch in the AST lists all global symbols.
The primary algorithm of the back-end compiler is to traverse the tree (high-level intermediate representation), and create an internal object code representation. There is an example of how to do this in MachineIndependent/intermOut.cpp.
Reduction of the tree to a linear byte-code style low-level intermediate representation is likely a good way to generate fully optimized code.
There is currently some dead old-style linker-type code still lying around.
Memory pool: parsing uses types derived from C++ std types, using a custom allocator that puts them in a memory pool. This makes allocation of individual container/contents just few cycles and deallocation free. This pool is popped after the AST is made and processed.
The use is simple: if you are going to call new, there are three cases:
- the object comes from the pool (its base class has the macro POOL_ALLOCATOR_NEW_DELETE in it) and you do not have to call delete
- it is a TString, in which case call NewPoolTString(), which gets it from the pool, and there is no corresponding delete
- the object does not come from the pool, and you have to do normal C++ memory management of what you new