Amber is a multi-API shader test framework. Graphics and compute bugs can be captured and communicated through a scripting interface. This removes the need to program to the low level interface when reproducing bugs.
Amber is broken into multiple layers: the applications, the parsing components and the script execution.
There are currently two applications, the [amber](../samples/amber.cc) application and the Amber integration into the Vulkan Conformance Test Suite ‘CTS’. These applications are responsible for configuring the script execution environment (setting up Vulkan, Dawn or another engine), calling into the parsing code to generate a test script and then passing that script into the script execution component.
We require the application to configure the execution engine. This allows the application to handle loading function pointers, doing configuration and other setups as required. There are no hardcoded assumptions in Amber as to how you're setting up the API to be tested.
This engine configuration is done through the VulkanEngineConfig in amber/amber_vulkan.h or the DawnEngineConfig in amber/amber_dawn.h.
The sample application does this configuration through the config_helper classes. samples/config_helper_vulkan.* and samples/config_helper_dawn.*. For the CTS, the Vulkan engine is already configured and we just set the VulkanEngineConfig as needed.
Accessing Amber itself is done through the Amber class in amber/amber.h. We require the application to parse and execute a script as two separate steps. This separation allows for the shaders to be retrieved after a parse command and then pre-compiled and passed into the execution. Passing the compiled shaders is optional.
A delegate object can be provided to Amber, for observing internal operations as the script is executed. The delegate can be a null pointer, to indicate no observations are requested.
If the delegate is provided and the LogGraphicsCalls method returns true then the Vulkan API wrappers will call Log for each call into Vulkan. If the LogGraphicsCallsTime also returns true then timings for those Vulkan calls will also be recorded. The timestamps are retrieved from the GetTimestampNS callback.
Amber can be instructed to retrieve the contents of buffers when execution is complete. This is done through the extractions list in the Amber Options structure. You must set the buffer_name for each of the buffers you want to extract. When Amber completes it will fill out the width, height and set of Value objects for that buffer.
There are two methods to execute a parsed script: Execute and ExecuteWithShaderData. They both accept the Recipe and Options, the ExecuteWithShaderData also accepts a map of shader name to data. The data is the compiled SPIR-V binary for the shader. This allows you to compile and cache the shader if needed.
Amber can use scripts written in two dialects: AmberScript, and VkScript. The AmberScript parser will be used if the first 7 characters of the script are #!amber, otherwise the VkScript parser will be used. The parsers both generate a Script which is our representation of the script file.
The AmberScript format maps closely to the format stored in the script objects. As such, there is a single Parser class for AmberScript which produces all of the needed script components.
For VkScript we do a bit of work to make the script match AmberScript. A default pipeline is generated and all content in the script is considered part of the generated pipeline. We generate names for all of the buffers in the file. The framebuffer is named framebuffer. The generated depth buffer is named depth_buffer. For other buffers, we generate a name of AutoBuf-<num> where the number is the current number of buffers seen counting from 0.
The VkScript parser is broken into three major chunks. The Parser, SectionParser and CommandParser. The Parser is the overall driver for the parser. The SectionParser breaks the input file into the overall chunks (each of the sections separated by the [blocks]). The CommandParser parses the [test] section specifically. For other sections they're parsed directly in the Parser object.
The Script object owns all of the pipelines, buffers, shaders, and command objects. Other objects hold pointers but the script holds the unique_ptr for these objects.
+--------+ +--------------+
| Script |---------------->| Requirements |
+--------+ +--------------+
|
+------------------+------------+--------+----------------------+
| | | |
v v v v
+---------+ +---------+ +---------+ +---------+
| +--------+ | +---------+ | +----------+ | +---------+
+--| +--------+ +--| +--------+ +--| +----------+ +--| +---------+
+--| Shader | +--| Buffer | +--| Pipeline | +--| Command |
+--------+ +--------+ +----------+ +---------+
^ ^ ^ | | ^ | |
| | | | | +---------------+ |
Entry point | | +-----------------+ | |
Optimizations | | Descriptor Set | |
Type | | Binding | |
Compiled Shader| | Attachment Location | |
| | | |
+------------------------------------------+ |
| |
+-----------------------------------------------+
BufferCommand
A Script contains shaders, pipelines, buffers, and commands. Pipelines contain shaders and buffers. An Amber buffer corresponds to either a buffer resource or an image resource in the backend engine's API. Script execution assumes that after executing a command, each Buffer object has a reference to the latest data for that buffer or image copied into the buffers memory. This means that a draw command will need to copy buffer data to the device, execute the draw, and then copy the device data back into the buffer. Amber does not do any extra calls to fill the buffers. Then engine must keep that data in sync.
The Pipeline object holds the context for a given set of tests. This includes shaders, colour attachments, depth/stencil attachment, data buffers, etc. The colour attachments will have their Attachment Location while data buffers will have a Descriptor Set and Binding provided.
When the script is executed the pipeline shaders will be compiled, if not provided through the shader map. This will fill in the Compiled Shader data in the each pipeline shader object. The CreatePipeline call will then be executed for a given pipeline.
With the pipelines created the Command objects will be executed. Each Command knows the Amber Pipeline associated with it, that Pipeline pointer is provided in the command execution and can be used to lookup the engine-specific representation of a pipeline.
When the Probe and ProbeSSBO commands are encountered they are not passed to the engine but sent to the Verifier. This will validate that the data in the specified buffer matches the test data. As mentioned above, this assumes that the Amber Buffer is kept up to date with the current device memory as we do not attempt to fill in the buffers before executing the Probe* calls.