Google Git
Sign in
android / platform / external / shaderc / glslang / refs/tags/ndk-r16-beta1
tag012fc1269c48168be1947977e4b450b2701bf90c
taggerThe Android Open Source Project <initial-contribution@android.com>Tue Nov 14 16:36:12 2017 -0800
objectf65b8dc5779b7124e4d471fdeb06187ead50274e 
ndk-r16-beta1
commitf65b8dc5779b7124e4d471fdeb06187ead50274e[log] [tgz]
authorDavid Neto <dneto@google.com>Tue Aug 01 11:53:59 2017 -0400
committerDavid Neto <dneto@google.com>Tue Aug 01 11:54:32 2017 -0400
tree51fbca3f0c85efc60abb63079892935d175fd0f6
parent7012bf923fe5493014479a0627f11d7316ac2008 [diff]
parentfda6edcbad862a2cb310da53f53e37db0fa270db [diff]
Merge remote-tracking branch 'aosp/upstream-master' into update-shaderc

Includes:
fda6edcb HLSL Tests: Fix two tests to be valid under FXC.
318a379b Non-functional: HLSL further simplications to base I/O flattening on.
8bcdf2ea Non-functional: HLSL: clean up dead code for splitting.
48dc5872 Merge pull request #1005 from LoopDawg/remove-unused
7a3cef10 HLSL: Non-functional: warning fix, remove unused member.
a58cc9ff GLSL reflection: Fix #985: reflect runtime sized arrays having no constant index.
a353bf1f Nonfunctional: Add reflect test case, and fix long lines in reflection code.
2ceec681 Nonfunctional: Shorten some lines to the coding standard, to retrigger failed test run.
38151b2f Merge pull request #1002 from amdrexu/bugfix
286ca432 SPV: Memory qualifiers should decorate top-level block members
f1f5058a Merge pull request #997 from LoopDawg/clipcull-semantic
307b6507 HLSL: handle multiple clip/cull semantic IDs
d2d3a142 SPV: Update to latest 1.0 headers, removing redundancies in GLSL.ext.AMD.h.
cd52fd5a Merge pull request #1000 from LoopDawg/samplecmpzero-cubearray-fix
ef94b1a5 Fix dref explicit LOD form of sample with cube texture arrays
53863a3a GLSL: Implement version 320 for ES.
9353f1af GLSL: Add version-number checking.
67eb4970 SPV/OpenGL: Require locations on non-opaque uniform variables.
ab008675 Merge pull request #991 from LoopDawg/resource-set-binding-fix
a2b71902 Reflection: Fix #977: Expose getBinding(), use in new getUniformBinding().
52017192 Fix crash with --resource-set-binding [n] (common set form, not per-register form)
0e392aa9 Merge pull request #996 from KhronosGroup/decorate-parameters
961cd35b SPV: Fix #995: Include memory decorations on parameters.
fad62972 SPV: Non-functional: support lists of decorations per parameter.
198652a6 Merge pull request #994 from KhronosGroup/opaque-init
0e6e2ffd Fix #980: flatten opaque initializers to use aliases.
37c202aa Merge pull request #992 from polarina/null-deref-fix
b5d9c11e Fix NULL pointer dereference in TParseContext::builtInOpCheck
91e69c03 SPV: Address #989: Don't add Location to built-ins when automapping.
56d2b990 Merge pull request #987 from KhronosGroup/global-std140
735d7e56 Address part A of #982: $Global will use std140 instead of HLSL offsets.
1e4a721c Merge pull request #978 from LoopDawg/global-const-init-fix
d854ac8b Merge pull request #983 from LoopDawg/warning-fix
b97b25e4 Fix StandAlone.cpp compilation warnings
0fca0baf WIP: HLSL: support global const initializers from non-constant rvalues
652db16f Build: Fix #975: shut up a compiler warning (code was fine).
4b1a890b Nonfunctional, infrastructure: Clean up Test directory droppings.
645fdaa7 Merge pull request #971 from amdrexu/bugfix
57e65929 HLSL: Translate directive [flatten] and [branch] to SPV control mask.
423fae48 Tests: Add a test for concurrent use of std430 and push_constant.
9645f782 Merge pull request #965 from chaoc/spv-khr-post-depth-coverage
c1204527 Implement SPV_KHR_post_depth_coverage
33bf7b2f Merge pull request #974 from LoopDawg/anyall-types
54b9ff9c HLSL: handle type conversion for any/all intrinsics
17b5f917 Merge pull request #969 from d3x0r/patch-1
3f70d405 Merge pull request #970 from rohith10/master
6206091e Fix CMake scripts: The set_property script can be used to set only a single property, so now setting the POSITION_INDEPENDENT_CODE property correctly.
45933124 Update CMakeLists.txt
4e53d905 Update CMakeLists.txt
89de217f Update CMakeLists.txt
9c6ea324 Update CMakeLists.txt
57a2b22d Update CMakeLists.txt
cde46127 Update CMakeLists.txt
02ed9eb2 Update CMakeLists.txt
42b69261 Update CMakeLists.txt
f8f494ff Add option to skip installation
eb5f12d1 Merge pull request #967 from dsrbecky/multiview
d1be7545 HLSL: Non-functional: Move partial flattened access into symbol node.
02a14e7c HLSL: Non-functional: some coding convention tweaks (120 columns, nullptr).
0af795e8 Merge pull request #966 from TiemoJung/io_remapper_update
aad93a80 Merge pull request #959 from TiemoJung/texture_upgrade
5513d9d0 Multiview extension: Accept layout(num_views) qualifier
0422eb23 io resolver improvements
99466020 Merge pull request #964 from dsrbecky/multiview
0fbe02c6 Implement extensions GL_OVR_multiview and GL_OVR_multiview2
fde4975d Fix for not transforming all image symbols into sampled images symbols
2ae23ca1 Merge pull request #957 from amdrexu/feature
37d7cfb1 Merge pull request #961 from benvanik/fix_order
82da44d4 Fixing initialization order to fix -Wreorder warning in clang.
89400ea7 GLSL: validation of early_fragment_tests, others, on an object.
37cdceed Implement extension GL_ARB_shader_stencil_export
89f8d1e6 HLSL: Fix #942: Map SV_TargetN to SPV Location N.
4f54c0c4 HLSL: Fix hull-shader test.
da9eb815 HLSL: Don't pin down which tessellation stage must error check input primitive.
fe6689c6 HLSL: support point mode.
102328b7 Merge pull request #950 from dsrbecky/shadow_samplers
a8a83204 Merge pull request #948 from KhronosGroup/env-control
6353d55e Command-line: Add support for setting language's environment.
c178f0a4 Infrastructure: Non-functional: rationalize some command-line processing.
4be4aebd Infrastructure: Non-functional: Move to rich description of environment.
c6ac40a1 Add support for GL_EXT_shadow_samplers
4fbb8cb4 Merge pull request #947 from LoopDawg/clip-cull-distance
c44b95fd WIP: HLSL: handle clip/cull distance array semantic matching
4329d555 HLSL: Broaden solution for #940, editing integer input for 'flat'.
f0bc598d HLSL: Force flat interpolation for structure members. Fixes #940.
be1085cb Build: add switch default to make compilers happy
fba125a9 Merge pull request #943 from xxxbxxx/for-upstream-1
54596ff9 HLSL: Force flat (nointerp) onto integer fragment inputs.
b1d97537 hlsl: "in out" is also an inout qualifier.
f02c8e6b Non-functional: Attempt to reset Travis error, while adding more nullptr use.
2fcdd64e HLSL: Remove support for named tbuffer/cbuffer.  Fixes #939.
054378d9 HLSL: Non-functional: Make test valid HLSL, and related comments/cleanup.
e2ff404f Merge pull request #917 from KhronosGroup/remove-redundant-locations
f790b161 Tests: Fix missing test result.
a931366f Standalone: Implement -D and -U for preprocessor macros.
04acb1b7 Standalone: Rationalize ShaderCompUnit and file data.
6263fb19 Infrastructure: Rationalize command-line options.
2d46e73b Merge pull request #931 from LoopDawg/scalar-mat-assign
e2713125 HLSL: fix several issues in mat construction from scalars
f7cd88a2 Merge pull request #937 from amdrexu/bugfix
301a2bc8 SPV: Fix an typo of SPV_AMD_texture_gather_bias_lod
82ae8c31 HLSL: Fix #924: Convert between two different arrays with cast.
0320d090 HLSL: Recognize types declared as identifiers as identifiers.
7cdf3fc3 Replace #422: Remove the redundant location setting in AST->SPIR-V.

Test: ndk/checkpoint.py on Linux; unit tests on Windows
Change-Id: I0260b403a61da54e9e655322e97b2af8a80c9760
tree: 51fbca3f0c85efc60abb63079892935d175fd0f6
  1. External/
  2. glslang/
  3. gtests/
  4. hlsl/
  5. OGLCompilersDLL/
  6. SPIRV/
  7. StandAlone/
  8. Test/
  9. .appveyor.yml
  10. .clang-format
  11. .gitattributes
  12. .gitignore
  13. .travis.yml
  14. ChooseMSVCCRT.cmake
  15. CMakeLists.txt
  16. make-revision
  17. README-spirv-remap.txt
  18. README.md
README.md

Also see the Khronos landing page for glslang as a reference front end:

https://www.khronos.org/opengles/sdk/tools/Reference-Compiler/

The above page includes where to get binaries, and is kept up to date regarding the feature level of glslang.

glslang

Build Status Build status

An OpenGL and OpenGL ES shader front end and validator.

There are several components:

  1. A GLSL/ESSL front-end for reference validation and translation of GLSL/ESSL into an AST.

  2. An HLSL front-end for translation of a broad generic HLL into the AST. See issue 362 and issue 701 for current status.

  3. A SPIR-V back end for translating the AST to SPIR-V.

  4. A standalone wrapper, glslangValidator, that can be used as a command-line tool for the above.

How to add a feature protected by a version/extension/stage/profile: See the comment in glslang/MachineIndependent/Versions.cpp.

Tasks waiting to be done are documented as GitHub issues.

Execution of Standalone Wrapper

To use the standalone binary form, execute glslangValidator, and it will print a usage statement. Basic operation is to give it a file containing a shader, and it will print out warnings/errors and optionally an AST.

The applied stage-specific rules are based on the file extension:

  • .vert for a vertex shader
  • .tesc for a tessellation control shader
  • .tese for a tessellation evaluation shader
  • .geom for a geometry shader
  • .frag for a fragment shader
  • .comp for a compute shader

There is also a non-shader extension

  • .conf for a configuration file of limits, see usage statement for example

Building

Dependencies

  • CMake: for generating compilation targets.
  • bison: optional, but needed when changing the grammar (glslang.y).
  • googletest: optional, but should use if making any changes to glslang.

Build steps

1) Check-Out this project

cd <parent of where you want glslang to be>
# If using SSH
git clone git@github.com:KhronosGroup/glslang.git
# Or if using HTTPS
git clone https://github.com/KhronosGroup/glslang.git

2) Check-Out External Projects

cd <the directory glslang was cloned to, "External" will be a subdirectory>
git clone https://github.com/google/googletest.git External/googletest

3) Configure

Assume the source directory is $SOURCE_DIR and the build directory is $BUILD_DIR:

For building on Linux (assuming using the Ninja generator):

cd $BUILD_DIR

cmake -GNinja -DCMAKE_BUILD_TYPE={Debug|Release|RelWithDebInfo} \
      -DCMAKE_INSTALL_PREFIX=`pwd`/install $SOURCE_DIR

For building on Windows:

cmake $SOURCE_DIR -DCMAKE_INSTALL_PREFIX=`pwd`/install
# The CMAKE_INSTALL_PREFIX part is for testing (explained later).

The CMake GUI also works for Windows (version 3.4.1 tested).

4) Build and Install

# for Linux:
ninja install

# for Windows:
cmake --build . --config {Release|Debug|MinSizeRel|RelWithDebInfo} \
      --target install

If using MSVC, after running CMake to configure, use the Configuration Manager to check the INSTALL project.

If you need to change the GLSL grammar

The grammar in glslang/MachineIndependent/glslang.y has to be recompiled with bison if it changes, the output files are committed to the repo to avoid every developer needing to have bison configured to compile the project when grammar changes are quite infrequent. For windows you can get binaries from GnuWin32.

The command to rebuild is:

bison --defines=MachineIndependent/glslang_tab.cpp.h
      -t MachineIndependent/glslang.y
      -o MachineIndependent/glslang_tab.cpp

The above command is also available in the bash script at glslang/updateGrammar.

Testing

Right now, there are two test harnesses existing in glslang: one is Google Test, one is the runtests script. The former runs unit tests and single-shader single-threaded integration tests, while the latter runs multiple-shader linking tests and multi-threaded tests.

Running tests

The runtests script requires compiled binaries to be installed into $BUILD_DIR/install. Please make sure you have supplied the correct configuration to CMake (using -DCMAKE_INSTALL_PREFIX) when building; otherwise, you may want to modify the path in the runtests script.

Running Google Test-backed tests:

cd $BUILD_DIR

# for Linux:
ctest

# for Windows:
ctest -C {Debug|Release|RelWithDebInfo|MinSizeRel}

# or, run the test binary directly
# (which gives more fine-grained control like filtering):
<dir-to-glslangtests-in-build-dir>/glslangtests

Running runtests script-backed tests:

cd $SOURCE_DIR/Test && ./runtests

Contributing tests

Test results should always be included with a pull request that modifies functionality.

If you are writing unit tests, please use the Google Test framework and place the tests under the gtests/ directory.

Integration tests are placed in the Test/ directory. It contains test input and a subdirectory baseResults/ that contains the expected results of the tests. Both the tests and baseResults/ are under source-code control.

Google Test runs those integration tests by reading the test input, compiling them, and then compare against the expected results in baseResults/. The integration tests to run via Google Test is registered in various gtests/*.FromFile.cpp source files. glslangtests provides a command-line option --update-mode, which, if supplied, will overwrite the golden files under the baseResults/ directory with real output from that invocation. For more information, please check gtests/ directory's README.

For the runtests script, it will generate current results in the localResults/ directory and diff them against the baseResults/. When you want to update the tracked test results, they need to be copied from localResults/ to baseResults/. This can be done by the bump shell script.

You can add your own private list of tests, not tracked publicly, by using localtestlist to list non-tracked tests. This is automatically read by runtests and included in the diff and bump process.

Programmatic Interfaces

Another piece of software can programmatically translate shaders to an AST using one of two different interfaces:

  • A new C++ class-oriented interface, or
  • The original C functional interface

The main() in StandAlone/StandAlone.cpp shows examples using both styles.

C++ Class Interface (new, preferred)

This interface is in roughly the last 1/3 of ShaderLang.h. It is in the glslang namespace and contains the following.

const char* GetEsslVersionString();
const char* GetGlslVersionString();
bool InitializeProcess();
void FinalizeProcess();

class TShader
    bool parse(...);
    void setStrings(...);
    const char* getInfoLog();

class TProgram
    void addShader(...);
    bool link(...);
    const char* getInfoLog();
    Reflection queries

See ShaderLang.h and the usage of it in StandAlone/StandAlone.cpp for more details.

C Functional Interface (orignal)

This interface is in roughly the first 2/3 of ShaderLang.h, and referred to as the Sh*() interface, as all the entry points start Sh.

The Sh*() interface takes a “compiler” call-back object, which it calls after building call back that is passed the AST and can then execute a backend on it.

The following is a simplified resulting run-time call stack:

ShCompile(shader, compiler) -> compiler(AST) -> <back end>

In practice, ShCompile() takes shader strings, default version, and warning/error and other options for controlling compilation.

Basic Internal Operation

  • Initial lexical analysis is done by the preprocessor in MachineIndependent/Preprocessor, and then refined by a GLSL scanner in MachineIndependent/Scan.cpp. There is currently no use of flex.

  • Code is parsed using bison on MachineIndependent/glslang.y with the aid of a symbol table and an AST. The symbol table is not passed on to the back-end; the intermediate representation stands on its own. The tree is built by the grammar productions, many of which are offloaded into ParseHelper.cpp, and by Intermediate.cpp.

  • The intermediate representation is very high-level, and represented as an in-memory tree. This serves to lose no information from the original program, and to have efficient transfer of the result from parsing to the back-end. In the AST, constants are propogated and folded, and a very small amount of dead code is eliminated.

    To aid linking and reflection, the last top-level branch in the AST lists all global symbols.

  • The primary algorithm of the back-end compiler is to traverse the tree (high-level intermediate representation), and create an internal object code representation. There is an example of how to do this in MachineIndependent/intermOut.cpp.

  • Reduction of the tree to a linear byte-code style low-level intermediate representation is likely a good way to generate fully optimized code.

  • There is currently some dead old-style linker-type code still lying around.

  • Memory pool: parsing uses types derived from C++ std types, using a custom allocator that puts them in a memory pool. This makes allocation of individual container/contents just few cycles and deallocation free. This pool is popped after the AST is made and processed.

    The use is simple: if you are going to call new, there are three cases:

    • the object comes from the pool (its base class has the macro POOL_ALLOCATOR_NEW_DELETE in it) and you do not have to call delete

    • it is a TString, in which case call NewPoolTString(), which gets it from the pool, and there is no corresponding delete

    • the object does not come from the pool, and you have to do normal C++ memory management of what you new