| <!-- HTML header for doxygen 1.8.15--> |
| <!-- Remember to use version doxygen 1.8.15 +--> |
| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <html xmlns="http://www.w3.org/1999/xhtml"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> |
| <meta http-equiv="X-UA-Compatible" content="IE=9"/> |
| <meta name="generator" content="Doxygen 1.8.15"/> |
| <meta name="robots" content="NOINDEX, NOFOLLOW" /> <!-- Prevent indexing by search engines --> |
| <title>Compute Library: Library architecture</title> |
| <link href="tabs.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript" src="jquery.js"></script> |
| <script type="text/javascript" src="dynsections.js"></script> |
| <link href="navtree.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript" src="resize.js"></script> |
| <script type="text/javascript" src="navtreedata.js"></script> |
| <script type="text/javascript" src="navtree.js"></script> |
| <script type="text/javascript"> |
| /* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */ |
| $(document).ready(initResizable); |
| /* @license-end */</script> |
| <link href="search/search.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript" src="search/searchdata.js"></script> |
| <script type="text/javascript" src="search/search.js"></script> |
| <script type="text/x-mathjax-config"> |
| MathJax.Hub.Config({ |
| extensions: ["tex2jax.js"], |
| jax: ["input/TeX","output/HTML-CSS"], |
| }); |
| </script><script type="text/javascript" async="async" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js"></script> |
| <link href="doxygen.css" rel="stylesheet" type="text/css" /> |
| <link href="stylesheet.css" rel="stylesheet" type="text/css"/> |
| </head> |
| <body> |
| <div id="top"><!-- do not remove this div, it is closed by doxygen! --> |
| <div id="titlearea"> |
| <table cellspacing="0" cellpadding="0"> |
| <tbody> |
| <tr style="height: 56px;"> |
| <img alt="Compute Library" src="https://raw.githubusercontent.com/ARM-software/ComputeLibrary/gh-pages/ACL_logo.png" style="max-width: 100%;margin-top: 15px;margin-left: 10px"/> |
| <td style="padding-left: 0.5em;"> |
| <div id="projectname"> |
|  <span id="projectnumber">20.02.1</span> |
| </div> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <!-- end header part --> |
| <!-- Generated by Doxygen 1.8.15 --> |
| <script type="text/javascript"> |
| /* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */ |
| var searchBox = new SearchBox("searchBox", "search",false,'Search'); |
| /* @license-end */ |
| </script> |
| <script type="text/javascript" src="menudata.js"></script> |
| <script type="text/javascript" src="menu.js"></script> |
| <script type="text/javascript"> |
| /* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */ |
| $(function() { |
| initMenu('',true,false,'search.php','Search'); |
| $(document).ready(function() { init_search(); }); |
| }); |
| /* @license-end */</script> |
| <div id="main-nav"></div> |
| </div><!-- top --> |
| <div id="side-nav" class="ui-resizable side-nav-resizable"> |
| <div id="nav-tree"> |
| <div id="nav-tree-contents"> |
| <div id="nav-sync" class="sync"></div> |
| </div> |
| </div> |
| <div id="splitbar" style="-moz-user-select:none;" |
| class="ui-resizable-handle"> |
| </div> |
| </div> |
| <script type="text/javascript"> |
| /* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */ |
| $(document).ready(function(){initNavTree('architecture.xhtml','');}); |
| /* @license-end */ |
| </script> |
| <div id="doc-content"> |
| <!-- window showing the filter options --> |
| <div id="MSearchSelectWindow" |
| onmouseover="return searchBox.OnSearchSelectShow()" |
| onmouseout="return searchBox.OnSearchSelectHide()" |
| onkeydown="return searchBox.OnSearchSelectKey(event)"> |
| </div> |
| |
| <!-- iframe showing the search results (closed by default) --> |
| <div id="MSearchResultsWindow"> |
| <iframe src="javascript:void(0)" frameborder="0" |
| name="MSearchResults" id="MSearchResults"> |
| </iframe> |
| </div> |
| |
| <div class="PageDoc"><div class="header"> |
| <div class="headertitle"> |
| <div class="title">Library architecture </div> </div> |
| </div><!--header--> |
| <div class="contents"> |
| <div class="toc"><h3>Table of Contents</h3> |
| <ul><li class="level1"><a href="#S4_1_1">Core vs Runtime libraries</a></li> |
| <li class="level1"><a href="#S4_1_2">Thread-safety</a></li> |
| <li class="level1"><a href="#S4_2_windows_kernels_mt_functions">Windows, kernels, multi-threading and functions</a><ul><li class="level2"><a href="#S4_2_1_windows">Windows</a></li> |
| <li class="level2"><a href="#S4_2_2">Kernels</a></li> |
| <li class="level2"><a href="#S4_2_3">Multi-threading</a></li> |
| <li class="level2"><a href="#S4_2_4">Functions</a></li> |
| <li class="level2"><a href="#S4_4_1_cl_scheduler">OpenCL Scheduler and kernel library</a></li> |
| <li class="level2"><a href="#S4_4_2_events_sync">OpenCL events and synchronization</a></li> |
| <li class="level2"><a href="#S4_4_2_cl_neon">OpenCL / NEON interoperability</a></li> |
| </ul> |
| </li> |
| <li class="level1"><a href="#S4_5_algorithms">Algorithms</a></li> |
| <li class="level1"><a href="#S4_6_images_tensors">Images, padding, border modes and tensors</a><ul><li class="level2"><a href="#S4_6_1_padding_and_border">Padding and border modes</a><ul><li class="level3"><a href="#padding">Padding</a></li> |
| <li class="level3"><a href="#valid_region">Valid regions</a></li> |
| </ul> |
| </li> |
| <li class="level2"><a href="#S4_6_2_tensors">Tensors</a></li> |
| <li class="level2"><a href="#S4_6_3_description_conventions">Images and Tensors description conventions</a></li> |
| <li class="level2"><a href="#S4_6_4_working_with_objects">Working with Images and Tensors using iterators</a></li> |
| <li class="level2"><a href="#S4_6_5_sub_tensors">Sub-tensors</a></li> |
| </ul> |
| </li> |
| <li class="level1"><a href="#S4_7_memory_manager">MemoryManager</a><ul><li class="level2"><a href="#S4_7_1_memory_manager_components">MemoryGroup, MemoryPool and MemoryManager Components</a><ul><li class="level3"><a href="#S4_7_1_1_memory_group">MemoryGroup</a></li> |
| <li class="level3"><a href="#S4_7_1_2_memory_pool">MemoryPool</a></li> |
| <li class="level3"><a href="#S4_7_1_2_memory_manager_components">MemoryManager Components</a></li> |
| </ul> |
| </li> |
| <li class="level2"><a href="#S4_7_2_working_with_memory_manager">Working with the Memory Manager</a></li> |
| <li class="level2"><a href="#S4_7_3_memory_manager_function_support">Function support</a></li> |
| </ul> |
| </li> |
| <li class="level1"><a href="#S4_8_import_memory">Import Memory Interface</a></li> |
| <li class="level1"><a href="#S4_9_opencl_tuner">OpenCL Tuner</a></li> |
| <li class="level1"><a href="#S4_10_weights_manager">Weights Manager</a><ul><li class="level2"><a href="#S4_10_1_working_with_weights_manager">Working with the Weights Manager</a></li> |
| </ul> |
| </li> |
| <li class="level1"><a href="#S5_0_experimental">Experimental Features</a><ul><li class="level2"><a href="#S5_1_run_time_context">Run-time Context</a></li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| <div class="textblock"><h1><a class="anchor" id="S4_1_1"></a> |
| Core vs Runtime libraries</h1> |
| <p>The Core library is a low level collection of algorithms implementations, it is designed to be embedded in existing projects and applications:</p> |
| <ul> |
| <li>It doesn't allocate any memory (All the memory allocations/mappings have to be handled by the caller).</li> |
| <li>It doesn't perform any kind of multi-threading (but provide information to the caller about how the workload can be split).</li> |
| </ul> |
| <p>The Runtime library is a very basic wrapper around the Core library which can be used for quick prototyping, it is basic in the sense that:</p> |
| <ul> |
| <li>It allocates images and tensors by using standard malloc().</li> |
| <li>It multi-threads NEON code in a very basic way using a very simple pool of threads.</li> |
| <li>For OpenCL it uses the default <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml" title="Provides global access to a CL context and command queue.">CLScheduler</a> command queue for all mapping operations and kernels.</li> |
| </ul> |
| <p>For maximum performance, it is expected that the users would re-implement an equivalent to the runtime library which suits better their needs (With a more clever multi-threading strategy, load-balancing between NEON and OpenCL, etc.)</p> |
| <h1><a class="anchor" id="S4_1_2"></a> |
| Thread-safety</h1> |
| <p>Although the library supports multi-threading during workload dispatch, thus parallelizing the execution of the workload at multiple threads, the current runtime module implementation is not thread-safe in the sense of executing different functions from separate threads. This lies to the fact that the provided scheduling mechanism wasn't designed with thread-safety in mind. As it is true with the rest of the runtime library a custom scheduling mechanism can be re-implemented to account for thread-safety if needed and be injected as the library's default scheduler.</p> |
| <h1><a class="anchor" id="S4_2_windows_kernels_mt_functions"></a> |
| Windows, kernels, multi-threading and functions</h1> |
| <h2><a class="anchor" id="S4_2_1_windows"></a> |
| Windows</h2> |
| <p>A <a class="el" href="classarm__compute_1_1_window.xhtml">Window</a> represents a workload to execute, it can handle up to <a class="el" href="classarm__compute_1_1_dimensions.xhtml#a1b67d5b720119d50faa286c774579ecc">Coordinates::num_max_dimensions</a> dimensions. Each dimension is defined by a start, end and step.</p> |
| <p>It can split into subwindows as long as <em>all</em> the following rules remain true for all the dimensions:</p> |
| <ul> |
| <li>max[n].start() <= sub[n].start() < max[n].end()</li> |
| <li>sub[n].start() < sub[n].end() <= max[n].end()</li> |
| <li>max[n].<a class="el" href="namespacearm__compute_1_1test_1_1validation.xhtml#a75b73e17c4ebe901e44af3b2b9846ab3">step()</a> == sub[n].<a class="el" href="namespacearm__compute_1_1test_1_1validation.xhtml#a75b73e17c4ebe901e44af3b2b9846ab3">step()</a></li> |
| <li>(sub[n].start() - max[n].start()) % max[n].<a class="el" href="namespacearm__compute_1_1test_1_1validation.xhtml#a75b73e17c4ebe901e44af3b2b9846ab3">step()</a> == 0</li> |
| <li>(sub[n].end() - sub[n].start()) % max[n].<a class="el" href="namespacearm__compute_1_1test_1_1validation.xhtml#a75b73e17c4ebe901e44af3b2b9846ab3">step()</a> == 0</li> |
| </ul> |
| <h2><a class="anchor" id="S4_2_2"></a> |
| Kernels</h2> |
| <p>Each implementation of the <a class="el" href="classarm__compute_1_1_i_kernel.xhtml">IKernel</a> interface (base class of all the kernels in the core library) works in the same way:</p> |
| <p>OpenCL kernels:</p> |
| <div class="fragment"><div class="line"><span class="comment">// Initialize the CLScheduler with the default context and default command queue</span></div><div class="line"><span class="comment">// Implicitly initializes the CLKernelLibrary to use ./cl_kernels as location for OpenCL kernels files and sets a default device for which OpenCL programs are built.</span></div><div class="line"><a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a9b58d0eb9a2af8e6d7908695e1557d6c">CLScheduler::get</a>().<a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a46ecf9ef0fe80ba2ed35acfc29856b7d">default_init</a>();</div><div class="line"></div><div class="line">cl::CommandQueue q = <a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a9b58d0eb9a2af8e6d7908695e1557d6c">CLScheduler::get</a>().<a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a398a2582c746d28fc125487a44c9ed74">queue</a>();</div><div class="line"><span class="comment">//Create a kernel object:</span></div><div class="line">MyKernel kernel;</div><div class="line"><span class="comment">// Initialize the kernel with the input/output and options you want to use:</span></div><div class="line">kernel.configure( <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>, output, option0, option1);</div><div class="line"><span class="comment">// Retrieve the execution window of the kernel:</span></div><div class="line"><span class="keyword">const</span> Window& max_window = kernel.window();</div><div class="line"><span class="comment">// Run the whole kernel in the current thread:</span></div><div class="line">kernel.run( q, max_window ); <span class="comment">// Enqueue the kernel to process the full window on the default queue</span></div><div class="line"></div><div class="line"><span class="comment">// Wait for the processing to complete:</span></div><div class="line">q.finish();</div></div><!-- fragment --><p>NEON / CPP kernels:</p> |
| <div class="fragment"><div class="line"><span class="comment">//Create a kernel object:</span></div><div class="line">MyKernel kernel;</div><div class="line"><span class="comment">// Initialize the kernel with the input/output and options you want to use:</span></div><div class="line">kernel.configure( <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>, output, option0, option1);</div><div class="line"><span class="comment">// Retrieve the execution window of the kernel:</span></div><div class="line"><span class="keyword">const</span> Window& max_window = kernel.window();</div><div class="line"><span class="comment">// Run the whole kernel in the current thread:</span></div><div class="line">kernel.run( max_window ); <span class="comment">// Run the kernel on the full window</span></div></div><!-- fragment --><h2><a class="anchor" id="S4_2_3"></a> |
| Multi-threading</h2> |
| <p>The previous section shows how to run a NEON / CPP kernel in the current thread, however if your system has several CPU cores, you will probably want the kernel to use several cores. Here is how this can be done:</p> |
| <div class="fragment"><div class="line">ThreadInfo <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>;</div><div class="line"><a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.cpu_info = &_cpu_info;</div><div class="line"></div><div class="line"><span class="keyword">const</span> Window &max_window = kernel->window();</div><div class="line"><span class="keyword">const</span> <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> num_iterations = max_window.num_iterations(split_dimension);</div><div class="line"><a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.num_threads = std::min(num_iterations, _num_threads);</div><div class="line"></div><div class="line"><span class="keywordflow">if</span>(num_iterations == 0)</div><div class="line">{</div><div class="line"> <span class="keywordflow">return</span>;</div><div class="line">}</div><div class="line"></div><div class="line"><span class="keywordflow">if</span>(!kernel->is_parallelisable() || <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.num_threads == 1)</div><div class="line">{</div><div class="line"> kernel->run(max_window, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>);</div><div class="line">}</div><div class="line"><span class="keywordflow">else</span></div><div class="line">{</div><div class="line"> <span class="keywordtype">int</span> t = 0;</div><div class="line"> <span class="keyword">auto</span> thread_it = _threads.begin();</div><div class="line"></div><div class="line"> <span class="keywordflow">for</span>(; t < <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.num_threads - 1; ++t, ++thread_it)</div><div class="line"> {</div><div class="line"> Window win = max_window.split_window(split_dimension, t, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.num_threads);</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.thread_id = t;</div><div class="line"> thread_it->start(kernel, win, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>);</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">// Run last part on main thread</span></div><div class="line"> Window win = max_window.split_window(split_dimension, t, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.num_threads);</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>.thread_id = t;</div><div class="line"> kernel->run(win, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a4f4125dba5283887b34f889b1c615c0c">info</a>);</div><div class="line"></div><div class="line"> <span class="keywordflow">try</span></div><div class="line"> {</div><div class="line"> <span class="keywordflow">for</span>(<span class="keyword">auto</span> &thread : _threads)</div><div class="line"> {</div><div class="line"> thread.wait();</div><div class="line"> }</div><div class="line"> }</div><div class="line"> <span class="keywordflow">catch</span>(<span class="keyword">const</span> std::system_error &e)</div><div class="line"> {</div><div class="line"> std::cerr << <span class="stringliteral">"Caught system_error with code "</span> << e.code() << <span class="stringliteral">" meaning "</span> << e.what() << <span class="charliteral">'\n'</span>;</div><div class="line"> }</div><div class="line">}</div></div><!-- fragment --><p>This is a very basic implementation which was originally used in the NEON runtime library by all the NEON functions.</p> |
| <dl class="section see"><dt>See also</dt><dd><a class="el" href="classarm__compute_1_1_c_p_p_scheduler.xhtml" title="C++11 implementation of a pool of threads to automatically split a kernel's execution among several t...">CPPScheduler</a></dd></dl> |
| <dl class="section note"><dt>Note</dt><dd>Some kernels like for example <a class="el" href="classarm__compute_1_1_n_e_histogram_kernel.xhtml">NEHistogramKernel</a> need some local temporary buffer to perform their calculations. In order to avoid memory corruption between threads, the local buffer must be of size: <code>memory_needed_per_thread * num_threads</code> and a unique thread_id between 0 and num_threads must be assigned to the <a class="el" href="structarm__compute_1_1_thread_info.xhtml">ThreadInfo</a> object passed to the <code>run</code> function.</dd></dl> |
| <h2><a class="anchor" id="S4_2_4"></a> |
| Functions</h2> |
| <p>Functions will automatically allocate the temporary buffers mentioned above, and will automatically multi-thread kernels' executions using the very basic scheduler described in the previous section.</p> |
| <p>Simple functions only call a single kernel (e.g <a class="el" href="classarm__compute_1_1_n_e_convolution3x3.xhtml">NEConvolution3x3</a>), while more complex ones consist of several kernels pipelined together (e.g <a class="el" href="classarm__compute_1_1_n_e_gaussian_pyramid.xhtml">NEGaussianPyramid</a>, <a class="el" href="classarm__compute_1_1_n_e_harris_corners.xhtml">NEHarrisCorners</a>). Check their documentation to find out which kernels are used by each function.</p> |
| <div class="fragment"><div class="line"><span class="comment">//Create a function object:</span></div><div class="line">MyFunction <span class="keyword">function</span>;</div><div class="line"><span class="comment">// Initialize the function with the input/output and options you want to use:</span></div><div class="line"><span class="keyword">function</span>.configure( <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>, output, option0, option1);</div><div class="line"><span class="comment">// Execute the function:</span></div><div class="line"><span class="keyword">function</span>.run();</div></div><!-- fragment --><dl class="section warning"><dt>Warning</dt><dd>The Compute Library requires Mali OpenCL DDK r8p0 or higher (OpenCL kernels are compiled using the -cl-arm-non-uniform-work-group-size flag)</dd></dl> |
| <dl class="section note"><dt>Note</dt><dd>All OpenCL functions and objects in the runtime library use the command queue associated with <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml" title="Provides global access to a CL context and command queue.">CLScheduler</a> for all operations, a real implementation would be expected to use different queues for mapping operations and kernels in order to reach a better GPU utilization.</dd></dl> |
| <h2><a class="anchor" id="S4_4_1_cl_scheduler"></a> |
| OpenCL Scheduler and kernel library</h2> |
| <p>The Compute Library runtime uses a single command queue and context for all the operations.</p> |
| <p>The user can get / set this context and command queue through <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml" title="Provides global access to a CL context and command queue.">CLScheduler</a>'s interface.</p> |
| <p>The user can get / set the target GPU device through the <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml" title="Provides global access to a CL context and command queue.">CLScheduler</a>'s interface.</p> |
| <dl class="section attention"><dt>Attention</dt><dd>Make sure the application is using the same context as the library as in OpenCL it is forbidden to share objects across contexts. This is done by calling <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml#a89be78d1ed13202968d25524b2d69393">CLScheduler::init()</a> or <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml#a46ecf9ef0fe80ba2ed35acfc29856b7d">CLScheduler::default_init()</a> at the beginning of your application.</dd> |
| <dd> |
| Make sure the scheduler's target is not changed after function classes are created.</dd></dl> |
| <p>All OpenCL kernels used by the library are built and stored in <a class="el" href="classarm__compute_1_1_c_l_kernel_library.xhtml">CLKernelLibrary</a>. If the library is compiled with embed_kernels=0 the application can set the path to the OpenCL kernels by calling <a class="el" href="classarm__compute_1_1_c_l_kernel_library.xhtml#a9f976367edcd9ab787375373e050b94b">CLKernelLibrary::init()</a>, by default the path is set to "./cl_kernels"</p> |
| <h2><a class="anchor" id="S4_4_2_events_sync"></a> |
| OpenCL events and synchronization</h2> |
| <p>In order to block until all the jobs in the <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml" title="Provides global access to a CL context and command queue.">CLScheduler</a>'s command queue are done executing the user can call <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml#ad55f80ed3cd8b6c4f247763b747016af">CLScheduler::sync()</a> or create a sync event using <a class="el" href="classarm__compute_1_1_c_l_scheduler.xhtml#a6096d26e8e29e1a7f01b0f083fb7e33f">CLScheduler::enqueue_sync_event()</a></p> |
| <p>For example: </p><div class="fragment"><div class="line"> PPMLoader ppm;</div><div class="line"> constexpr <span class="keywordtype">int</span> scale_factor = 2;</div><div class="line"></div><div class="line"> <a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a9b58d0eb9a2af8e6d7908695e1557d6c">CLScheduler::get</a>().<a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a46ecf9ef0fe80ba2ed35acfc29856b7d">default_init</a>();</div><div class="line"></div><div class="line"> <span class="keywordflow">if</span>(argc < 2)</div><div class="line"> {</div><div class="line"> <span class="comment">// Print help</span></div><div class="line"> std::cout << <span class="stringliteral">"Usage: ./build/cl_events [input_image.ppm]\n\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">"No input_image provided, creating a dummy 640x480 image\n"</span>;</div><div class="line"> <span class="comment">// Create an empty grayscale 640x480 image</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->init(TensorInfo(640, 480, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>));</div><div class="line"> }</div><div class="line"> <span class="keywordflow">else</span></div><div class="line"> {</div><div class="line"> ppm.open(argv[1]);</div><div class="line"> ppm.init_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>);</div><div class="line"> }</div><div class="line"></div><div class="line"> TensorInfo dst_info(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info()->dimension(0) / scale_factor, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info()->dimension(1) / scale_factor, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>);</div><div class="line"></div><div class="line"> <span class="comment">// Configure the temporary and destination images</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_i_tensor_allocator.xhtml#af36143939a43fa124312e395975091ed">init</a>(dst_info);</div><div class="line"> tmp_scale_median.allocator()->init(dst_info);</div><div class="line"> tmp_median_gauss.allocator()->init(dst_info);</div><div class="line"></div><div class="line"> <span class="comment">//Configure the functions:</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#acec6d8ad52a28972fa74e071c1a63b6a">scale</a>.configure(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, &tmp_scale_median, <a class="code" href="namespacearm__compute.xhtml#a966a9c417ce5e94dca08d9b5e745c0c9a7f5ccbc3d30c2cd3fd04d567946cbde2">InterpolationPolicy::NEAREST_NEIGHBOR</a>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a>);</div><div class="line"> median.configure(&tmp_scale_median, &tmp_median_gauss, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a>);</div><div class="line"> gauss.configure(&tmp_median_gauss, &<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a>);</div><div class="line"></div><div class="line"> <span class="comment">// Allocate all the images</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->allocate();</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_c_l_tensor_allocator.xhtml#a6e509c2a177b0b29e9e2369535094dee">allocate</a>();</div><div class="line"> tmp_scale_median.allocator()->allocate();</div><div class="line"> tmp_median_gauss.allocator()->allocate();</div><div class="line"></div><div class="line"> <span class="comment">// Fill the input image with the content of the PPM image if a filename was provided:</span></div><div class="line"> <span class="keywordflow">if</span>(ppm.is_open())</div><div class="line"> {</div><div class="line"> ppm.fill_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>);</div><div class="line"> output_filename = std::string(argv[1]) + <span class="stringliteral">"_out.ppm"</span>;</div><div class="line"> }</div></div><!-- fragment --> <h2><a class="anchor" id="S4_4_2_cl_neon"></a> |
| OpenCL / NEON interoperability</h2> |
| <p>You can mix OpenCL and NEON kernels and functions. However it is the user's responsibility to handle the mapping/unmapping of OpenCL objects, for example:</p> |
| <div class="fragment"><div class="line"> PPMLoader ppm;</div><div class="line"></div><div class="line"> <a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a9b58d0eb9a2af8e6d7908695e1557d6c">CLScheduler::get</a>().<a class="code" href="classarm__compute_1_1_c_l_scheduler.xhtml#a46ecf9ef0fe80ba2ed35acfc29856b7d">default_init</a>();</div><div class="line"></div><div class="line"> <span class="keywordflow">if</span>(argc < 2)</div><div class="line"> {</div><div class="line"> <span class="comment">// Print help</span></div><div class="line"> std::cout << <span class="stringliteral">"Usage: ./build/cl_convolution [input_image.ppm]\n\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">"No input_image provided, creating a dummy 640x480 image\n"</span>;</div><div class="line"> <span class="comment">// Create an empty grayscale 640x480 image</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->init(TensorInfo(640, 480, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>));</div><div class="line"> }</div><div class="line"> <span class="keywordflow">else</span></div><div class="line"> {</div><div class="line"> ppm.open(argv[1]);</div><div class="line"> ppm.init_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>);</div><div class="line"> }</div><div class="line"></div><div class="line"> TensorInfo scale_median_info(TensorInfo(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info()->dimension(0) / 2, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info()->dimension(1) / 2, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>));</div><div class="line"></div><div class="line"> <span class="comment">// Configure the temporary and destination images</span></div><div class="line"> scale_median.allocator()->init(scale_median_info);</div><div class="line"> median_gauss.allocator()->init(scale_median_info);</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_i_tensor_allocator.xhtml#af36143939a43fa124312e395975091ed">init</a>(scale_median_info);</div><div class="line"></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#acec6d8ad52a28972fa74e071c1a63b6a">scale</a>.configure(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, &scale_median, <a class="code" href="namespacearm__compute.xhtml#a966a9c417ce5e94dca08d9b5e745c0c9a7f5ccbc3d30c2cd3fd04d567946cbde2">InterpolationPolicy::NEAREST_NEIGHBOR</a>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a>);</div><div class="line"> median.configure(&scale_median, &median_gauss, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a>);</div><div class="line"> gauss.configure(&median_gauss, &<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a>);</div><div class="line"></div><div class="line"> <span class="comment">// Allocate all the images</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->allocate();</div><div class="line"> scale_median.allocator()->allocate();</div><div class="line"> median_gauss.allocator()->allocate();</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_c_l_tensor_allocator.xhtml#a6e509c2a177b0b29e9e2369535094dee">allocate</a>();</div><div class="line"></div><div class="line"> <span class="comment">// Fill the input image with the content of the PPM image if a filename was provided:</span></div><div class="line"> <span class="keywordflow">if</span>(ppm.is_open())</div><div class="line"> {</div><div class="line"> ppm.fill_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>);</div><div class="line"> <span class="keyword">const</span> std::string output_filename = std::string(argv[1]) + <span class="stringliteral">"_out.ppm"</span>;</div><div class="line"> }</div></div><!-- fragment --> <dl class="section see"><dt>See also</dt><dd>main_neoncl_scale_median_gaussian</dd></dl> |
| <h1><a class="anchor" id="S4_5_algorithms"></a> |
| Algorithms</h1> |
| <p>All computer vision algorithms in this library have been implemented following the <a href="https://www.khronos.org/registry/vx/specs/1.1/html/">OpenVX 1.1 specifications</a>. Please refer to the Khronos documentation for more information.</p> |
| <h1><a class="anchor" id="S4_6_images_tensors"></a> |
| Images, padding, border modes and tensors</h1> |
| <p>Most kernels and functions in the library process images, however, in order to be future proof most of the kernels actually accept tensors. See below for more information about how they are related.</p> |
| <dl class="section attention"><dt>Attention</dt><dd>Each memory object can be written by only one kernel, however it can be read by several kernels. Writing to the same object from several kernels will result in undefined behavior. The kernel writing to an object must be configured before the kernel(s) reading from it.</dd></dl> |
| <h2><a class="anchor" id="S4_6_1_padding_and_border"></a> |
| Padding and border modes</h2> |
| <p>Several algorithms require a neighborhood around the current pixel to compute it's value. This means the algorithm will not be able to process the borders of the image unless you give it more information about how those border pixels should be processed. The <a class="el" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327">BorderMode</a> enum is used for this purpose.</p> |
| <p>You have 3 types of <a class="el" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327">BorderMode</a> :</p> |
| <ul> |
| <li><a class="el" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a0db45d2a4141101bdfe48e3314cfbca3">BorderMode::UNDEFINED</a> : Neighbor pixels outside of the image are treated as undefined. As a result all the pixels which are on the border will have a value which is undefined.</li> |
| <li><a class="el" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a4ef59320fbe90fe47d40f1f71e4c5daa">BorderMode::REPLICATE</a> : Neighbor pixels outside of the image are treated as having the same value as the closest valid pixel.</li> |
| <li><a class="el" href="namespacearm__compute.xhtml#a14d24d90ab4ba2956e92e27890ba4c91a8d6b5cada83510220f59e00ce86d4d92">BorderMode::CONSTANT</a> : Neighbor pixels outside of the image are treated as having the same constant value. (The user can choose what this value should be).</li> |
| </ul> |
| <p>Moreover both OpenCL and NEON use vector loads and stores instructions to access the data in buffers, so in order to avoid having special cases to handle for the borders all the images and tensors used in this library must be padded.</p> |
| <h3><a class="anchor" id="padding"></a> |
| Padding</h3> |
| <p>There are different ways padding can be calculated:</p> |
| <ul> |
| <li>Accurate padding:</li> |
| </ul> |
| <div class="fragment"><div class="line"> PPMLoader ppm;</div><div class="line"></div><div class="line"> <span class="keywordflow">if</span>(argc < 2)</div><div class="line"> {</div><div class="line"> <span class="comment">// Print help</span></div><div class="line"> std::cout << <span class="stringliteral">"Usage: ./build/neon_convolution [input_image.ppm]\n\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">"No input_image provided, creating a dummy 640x480 image\n"</span>;</div><div class="line"> <span class="comment">// Initialize just the dimensions and format of your buffers:</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->init(TensorInfo(640, 480, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>));</div><div class="line"> }</div><div class="line"> <span class="keywordflow">else</span></div><div class="line"> {</div><div class="line"> ppm.open(argv[1]);</div><div class="line"> <span class="comment">// Initialize just the dimensions and format of your buffers:</span></div><div class="line"> ppm.init_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>);</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">// Initialize just the dimensions and format of the temporary and destination images:</span></div><div class="line"> tmp.allocator()->init(*<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info());</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_i_tensor_allocator.xhtml#af36143939a43fa124312e395975091ed">init</a>(*<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info());</div><div class="line"></div><div class="line"> <span class="comment">// Apply a Gaussian 3x3 filter to the source image followed by a Gaussian 5x5:</span></div><div class="line"> <span class="comment">// The function will automatically update the padding information inside input and output to match its requirements</span></div><div class="line"> conv3x3.configure(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, &tmp, <a class="code" href="cl__convolution_8cpp.xhtml#a282820d9d081a04a78f602de4d41520a">gaussian3x3</a>.data(), 0 <span class="comment">/* Let arm_compute calculate the scale */</span>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a0db45d2a4141101bdfe48e3314cfbca3">BorderMode::UNDEFINED</a>);</div><div class="line"> conv5x5.configure(&tmp, &<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>, <a class="code" href="cl__convolution_8cpp.xhtml#a1ff25aea7e858eb83e7af7cefb60bb82">gaussian5x5</a>.data(), 0 <span class="comment">/* Let arm_compute calculate the scale */</span>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a0db45d2a4141101bdfe48e3314cfbca3">BorderMode::UNDEFINED</a>);</div><div class="line"></div><div class="line"> <span class="comment">// Now that the padding requirements are known we can allocate the images:</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->allocate();</div><div class="line"> tmp.allocator()->allocate();</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_c_l_tensor_allocator.xhtml#a6e509c2a177b0b29e9e2369535094dee">allocate</a>();</div><div class="line"></div><div class="line"> <span class="comment">// Fill the input image with the content of the PPM image if a filename was provided:</span></div><div class="line"> <span class="keywordflow">if</span>(ppm.is_open())</div><div class="line"> {</div><div class="line"> ppm.fill_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>);</div><div class="line"> output_filename = std::string(argv[1]) + <span class="stringliteral">"_out.ppm"</span>;</div><div class="line"> }</div></div><!-- fragment --> <dl class="section note"><dt>Note</dt><dd>It's important to call allocate <b>after</b> the function is configured: if the image / tensor is already allocated then the function will shrink its execution window instead of increasing the padding. (See below for more details).</dd></dl> |
| <ul> |
| <li>Manual padding / no padding / auto padding: You can allocate your images / tensors up front (before configuring your functions). In that case the function will use whatever padding is available and will shrink its execution window if there isn't enough padding available (which translates into a smaller valid region for the output). See also <a class="el" href="architecture.xhtml#valid_region">Valid regions</a>). If you don't want to manually set the padding but still want to allocate your objects upfront then you can use auto_padding. It guarantees that the allocation will have enough padding to run any of the provided functions.</li> |
| </ul> |
| <div class="fragment"><div class="line"><a class="code" href="struct_image.xhtml">Image</a> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>;</div><div class="line"></div><div class="line"><span class="comment">// Use auto padding for the input:</span></div><div class="line"><a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info()->init_auto_padding(TensorShape(640u,480u), <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>);</div><div class="line"></div><div class="line"><span class="comment">// Use manual padding for the destination image</span></div><div class="line"><a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#ad45f0c01a0713dfb6bd7232c7f396fc4">info</a>()-><a class="code" href="classarm__compute_1_1_tensor_info.xhtml#ad6b64f33be1e66dcf7612483ffb8fd63">init</a>(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.info()->tensor_shape(), <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a6669348b484e3008dca2bfa8e85e40b5">Format::U8</a>, strides_in_bytes, offset_first_element_in_bytes, total_size_in_bytes);</div><div class="line"></div><div class="line"><span class="comment">// Allocate all the images</span></div><div class="line"><a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>.allocator()->allocate();</div><div class="line"><a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>.<a class="code" href="classarm__compute_1_1_c_l_tensor.xhtml#a4083de30daebd6bdee6b35d9c8262108">allocator</a>()-><a class="code" href="classarm__compute_1_1_c_l_tensor_allocator.xhtml#a6e509c2a177b0b29e9e2369535094dee">allocate</a>();</div><div class="line"><span class="comment">// Fill the input image with the content of the PPM image if a filename was provided:</span></div><div class="line">fill_image(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>);</div><div class="line"></div><div class="line">NEGaussian3x3 gauss;</div><div class="line"></div><div class="line"><span class="comment">// Apply a Gaussian 3x3 filter to the source image (Note: if the padding provided is not enough then the execution window and valid region of the output will be shrunk)</span></div><div class="line">gauss.configure(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a989ab3e96426615bb98e04e0235088ca">src</a>, &<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#adbf67dcee294e673cf796f1ed8aeb6a4">dst</a>, <a class="code" href="namespacearm__compute.xhtml#a15a05537a472ee742404821851529327a0db45d2a4141101bdfe48e3314cfbca3">BorderMode::UNDEFINED</a>);</div><div class="line"></div><div class="line"><span class="comment">//Execute the functions:</span></div><div class="line">gauss.run();</div></div><!-- fragment --><dl class="section warning"><dt>Warning</dt><dd>Some kernels need up to 3 neighbor values to calculate the value of a given pixel. Therefore, to be safe, we use a 4-pixel padding all around the image. In addition, some kernels read and write up to 32 pixels at the same time. To cover that case as well we add an extra 32 pixels of padding at the end of each row. As a result auto padded buffers waste a lot of memory and are less cache friendly. It is therefore recommended to use accurate padding or manual padding wherever possible.</dd></dl> |
| <h3><a class="anchor" id="valid_region"></a> |
| Valid regions</h3> |
| <p>Some kernels (like edge detectors for example) need to read values of neighboring pixels to calculate the value of a given pixel, it is therefore not possible to calculate the values of the pixels on the edges.</p> |
| <p>Another case is: if a kernel processes 8 pixels per iteration and the image's dimensions are not a multiple of 8 and not enough padding is available then the kernel will not be able to process the pixels near the right edge. As a result these pixels will be left undefined.</p> |
| <p>In order to know which pixels have been calculated, each kernel sets a valid region for each output image or tensor. See also <a class="el" href="classarm__compute_1_1_tensor_info.xhtml#a3c20d908342087484d883574d55dd482">TensorInfo::valid_region()</a>, <a class="el" href="structarm__compute_1_1_valid_region.xhtml">ValidRegion</a></p> |
| <h2><a class="anchor" id="S4_6_2_tensors"></a> |
| Tensors</h2> |
| <p>Tensors are multi-dimensional arrays with a maximum of <a class="el" href="classarm__compute_1_1_dimensions.xhtml#a1b67d5b720119d50faa286c774579ecc">Coordinates::num_max_dimensions</a> dimensions.</p> |
| <p>Depending on the number of dimensions tensors can be interpreted as various objects. A scalar can be represented as a zero-dimensional tensor and a vector of numbers can be represented as an one-dimensional tensor. Further, an image is actually just a 2D tensor, a 3D tensor can be seen as an array of images and a 4D tensor as a 2D array of images, etc.</p> |
| <dl class="section note"><dt>Note</dt><dd>Most algorithms process images (i.e a 2D slice of the tensor), therefore only padding along the X and Y axes is required (2D slices can be stored contiguously in memory).</dd></dl> |
| <h2><a class="anchor" id="S4_6_3_description_conventions"></a> |
| Images and Tensors description conventions</h2> |
| <p><a class="el" href="struct_image.xhtml" title="Structure to hold Image information.">Image</a> objects are defined by a <a class="el" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58">Format</a> and dimensions expressed as [width, height, batch]</p> |
| <p>Tensors are defined by a <a class="el" href="namespacearm__compute.xhtml#ad8ed01ff3ff33333d8e19db4d2818bb6">DataType</a> plus a number of channels (Always expected to be 1 for now) and their dimensions are expressed as [width, height, feature_maps, batch].</p> |
| <p>In other words, the lower three dimensions of a tensor specify a single input in [width, height, feature_maps], while any other specified dimension represents a batch in the appropriate dimension space. For example, a tensor with dimensions [128, 128, 64, 16] represents a 1D batch space with 16 batches of 128 elements in width and height and 64 feature maps each. Each kernel specifies the expected layout of each of its tensors in its documentation.</p> |
| <dl class="section note"><dt>Note</dt><dd>Unless specified otherwise in the kernel's or function's documentation all tensors and images parameters passed must have identical dimensions.</dd> |
| <dd> |
| Unless specified otherwise in the kernel's or function's documentation the number of channels for tensors is expected to be 1 (For images, the number of channels is inferred from the <a class="el" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58">Format</a>).</dd></dl> |
| <dl class="section attention"><dt>Attention</dt><dd>Regardless of the <a class="el" href="namespacearm__compute.xhtml#ad8ed01ff3ff33333d8e19db4d2818bb6">DataType</a> used by a tensor the <a class="el" href="classarm__compute_1_1_i_tensor.xhtml#ab988210662dbd3bf32fd563c7dd1bdbf">ITensor::buffer()</a> method will always return a uint8_t pointer, and all the metadata in <a class="el" href="classarm__compute_1_1_tensor_info.xhtml">TensorInfo</a> will be expressed in bytes. It is the user's responsibility to cast the pointer to the correct type.</dd></dl> |
| <p>For example, to read the element located at the coordinates (x,y) of a float tensor:</p> |
| <div class="fragment"><div class="line"><span class="keywordtype">float</span> value = *reinterpret_cast<float*>(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>.buffer() + <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>.info()->offset_element_in_bytes(Coordinates(x,y)));</div></div><!-- fragment --><h2><a class="anchor" id="S4_6_4_working_with_objects"></a> |
| Working with Images and Tensors using iterators</h2> |
| <p>The library provides some iterators to access objects' data. Iterators are created by associating a data object (An image or a tensor for example) with an iteration window.</p> |
| <p>Iteration windows are defined by an array of dimensions, each of which consists of a start, end and step.</p> |
| <p>The <a class="el" href="namespacearm__compute.xhtml#a5002bf7ec46d52971f9526e94172cfee">execute_window_loop</a> function takes an execution window, a lambda function and one or more iterators. It will iterate through every element of the execution window and for each element it will update the iterators accordingly and call the lambda function.</p> |
| <p>Here are a couple of examples of how to use the iterators to fill / read tensors:</p> |
| <div class="fragment"><div class="line"> constexpr <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> width = 4;</div><div class="line"> constexpr <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> height = 3;</div><div class="line"> constexpr <span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> batch = 2;</div><div class="line"></div><div class="line"> src_data = <span class="keyword">new</span> <span class="keywordtype">float</span>[width * height * batch];</div><div class="line"> dst_data = <span class="keyword">new</span> <span class="keywordtype">float</span>[width * height * batch];</div><div class="line"></div><div class="line"> <span class="comment">// Fill src_data with dummy values:</span></div><div class="line"> <span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#aa76b4a6e74940dabc5b7fc6b2dab3545">b</a> = 0; <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#aa76b4a6e74940dabc5b7fc6b2dab3545">b</a> < batch; <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#aa76b4a6e74940dabc5b7fc6b2dab3545">b</a>++)</div><div class="line"> {</div><div class="line"> <span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> h = 0; h < height; h++)</div><div class="line"> {</div><div class="line"> <span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> <span class="keywordtype">int</span> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a1a367830ae09bf6138df822888ec1d71">w</a> = 0; <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a1a367830ae09bf6138df822888ec1d71">w</a> < width; <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a1a367830ae09bf6138df822888ec1d71">w</a>++)</div><div class="line"> {</div><div class="line"> src_data[<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#aa76b4a6e74940dabc5b7fc6b2dab3545">b</a> * (width * height) + h * width + <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a1a367830ae09bf6138df822888ec1d71">w</a>] = static_cast<float>(100 * <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#aa76b4a6e74940dabc5b7fc6b2dab3545">b</a> + 10 * h + <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a1a367830ae09bf6138df822888ec1d71">w</a>);</div><div class="line"> }</div><div class="line"> }</div><div class="line"> }</div><div class="line"></div><div class="line"> <span class="comment">// Initialize the tensors dimensions and type:</span></div><div class="line"> <span class="keyword">const</span> TensorShape <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a45cde9abb508c62d67c3bb2b9bf566a5">shape</a>(width, height, batch);</div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>.allocator()->init(TensorInfo(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a45cde9abb508c62d67c3bb2b9bf566a5">shape</a>, 1, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a44ad4ef5a76e6aa6fb3e3fa079a54fda">DataType::F32</a>));</div><div class="line"> output.allocator()->init(TensorInfo(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a45cde9abb508c62d67c3bb2b9bf566a5">shape</a>, 1, <a class="code" href="namespacearm__compute.xhtml#ab4e88c89b3b7ea1735996cc4def22d58a44ad4ef5a76e6aa6fb3e3fa079a54fda">DataType::F32</a>));</div><div class="line"></div><div class="line"> <span class="comment">// Configure softmax:</span></div><div class="line"> softmax.configure(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>, &output);</div><div class="line"></div><div class="line"> <span class="comment">// Allocate the input / output tensors:</span></div><div class="line"> <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>.allocator()->allocate();</div><div class="line"> output.allocator()->allocate();</div><div class="line"></div><div class="line"> <span class="comment">// Fill the input tensor:</span></div><div class="line"> <span class="comment">// Simplest way: create an iterator to iterate through each element of the input tensor:</span></div><div class="line"> Window input_window;</div><div class="line"> input_window.use_tensor_dimensions(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>.info()->tensor_shape());</div><div class="line"> std::cout << <span class="stringliteral">" Dimensions of the input's iterator:\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">" X = [start="</span> << input_window.x().start() << <span class="stringliteral">", end="</span> << input_window.x().end() << <span class="stringliteral">", step="</span> << input_window.x().step() << <span class="stringliteral">"]\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">" Y = [start="</span> << input_window.y().start() << <span class="stringliteral">", end="</span> << input_window.y().end() << <span class="stringliteral">", step="</span> << input_window.y().step() << <span class="stringliteral">"]\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">" Z = [start="</span> << input_window.z().start() << <span class="stringliteral">", end="</span> << input_window.z().end() << <span class="stringliteral">", step="</span> << input_window.z().step() << <span class="stringliteral">"]\n"</span>;</div><div class="line"></div><div class="line"> <span class="comment">// Create an iterator:</span></div><div class="line"> Iterator input_it(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a8fcf2ddd9a1d58b1b280f5c0aed71845">input</a>, input_window);</div><div class="line"></div><div class="line"> <span class="comment">// Iterate through the elements of src_data and copy them one by one to the input tensor:</span></div><div class="line"> <span class="comment">// This is equivalent to:</span></div><div class="line"> <span class="comment">// for( unsigned int z = 0; z < batch; ++z)</span></div><div class="line"> <span class="comment">// {</span></div><div class="line"> <span class="comment">// for( unsigned int y = 0; y < height; ++y)</span></div><div class="line"> <span class="comment">// {</span></div><div class="line"> <span class="comment">// for( unsigned int x = 0; x < width; ++x)</span></div><div class="line"> <span class="comment">// {</span></div><div class="line"> <span class="comment">// *reinterpret_cast<float*>( input.buffer() + input.info()->offset_element_in_bytes(Coordinates(x,y,z))) = src_data[ z * (width*height) + y * width + x];</span></div><div class="line"> <span class="comment">// }</span></div><div class="line"> <span class="comment">// }</span></div><div class="line"> <span class="comment">// }</span></div><div class="line"> <span class="comment">// Except it works for an arbitrary number of dimensions</span></div><div class="line"> <a class="code" href="namespacearm__compute.xhtml#a5002bf7ec46d52971f9526e94172cfee">execute_window_loop</a>(input_window, [&](<span class="keyword">const</span> Coordinates & <span class="keywordtype">id</span>)</div><div class="line"> {</div><div class="line"> std::cout << <span class="stringliteral">"Setting item ["</span> << <span class="keywordtype">id</span>.x() << <span class="stringliteral">","</span> << <span class="keywordtype">id</span>.y() << <span class="stringliteral">","</span> << <span class="keywordtype">id</span>.z() << <span class="stringliteral">"]\n"</span>;</div><div class="line"> *reinterpret_cast<float *>(input_it.ptr()) = src_data[<span class="keywordtype">id</span>.z() * (width * height) + <span class="keywordtype">id</span>.y() * width + <span class="keywordtype">id</span>.x()];</div><div class="line"> },</div><div class="line"> input_it);</div><div class="line"></div><div class="line"> <span class="comment">// More efficient way: create an iterator to iterate through each row (instead of each element) of the output tensor:</span></div><div class="line"> Window output_window;</div><div class="line"> output_window.use_tensor_dimensions(output.info()->tensor_shape(), <span class="comment">/* first_dimension =*/</span><a class="code" href="classarm__compute_1_1_window.xhtml#ad2d402364fa822b0b7775081291eeca9">Window::DimY</a>); <span class="comment">// Iterate through the rows (not each element)</span></div><div class="line"> std::cout << <span class="stringliteral">" Dimensions of the output's iterator:\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">" X = [start="</span> << output_window.x().start() << <span class="stringliteral">", end="</span> << output_window.x().end() << <span class="stringliteral">", step="</span> << output_window.x().step() << <span class="stringliteral">"]\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">" Y = [start="</span> << output_window.y().start() << <span class="stringliteral">", end="</span> << output_window.y().end() << <span class="stringliteral">", step="</span> << output_window.y().step() << <span class="stringliteral">"]\n"</span>;</div><div class="line"> std::cout << <span class="stringliteral">" Z = [start="</span> << output_window.z().start() << <span class="stringliteral">", end="</span> << output_window.z().end() << <span class="stringliteral">", step="</span> << output_window.z().step() << <span class="stringliteral">"]\n"</span>;</div><div class="line"></div><div class="line"> <span class="comment">// Create an iterator:</span></div><div class="line"> Iterator output_it(&output, output_window);</div><div class="line"></div><div class="line"> <span class="comment">// Iterate through the rows of the output tensor and copy them to dst_data:</span></div><div class="line"> <span class="comment">// This is equivalent to:</span></div><div class="line"> <span class="comment">// for( unsigned int z = 0; z < batch; ++z)</span></div><div class="line"> <span class="comment">// {</span></div><div class="line"> <span class="comment">// for( unsigned int y = 0; y < height; ++y)</span></div><div class="line"> <span class="comment">// {</span></div><div class="line"> <span class="comment">// memcpy( dst_data + z * (width*height) + y * width, input.buffer() + input.info()->offset_element_in_bytes(Coordinates(0,y,z)), width * sizeof(float));</span></div><div class="line"> <span class="comment">// }</span></div><div class="line"> <span class="comment">// }</span></div><div class="line"> <span class="comment">// Except it works for an arbitrary number of dimensions</span></div><div class="line"> <a class="code" href="namespacearm__compute.xhtml#a5002bf7ec46d52971f9526e94172cfee">execute_window_loop</a>(output_window, [&](<span class="keyword">const</span> Coordinates & <span class="keywordtype">id</span>)</div><div class="line"> {</div><div class="line"> std::cout << <span class="stringliteral">"Copying one row starting from ["</span> << <span class="keywordtype">id</span>.x() << <span class="stringliteral">","</span> << <span class="keywordtype">id</span>.y() << <span class="stringliteral">","</span> << <span class="keywordtype">id</span>.z() << <span class="stringliteral">"]\n"</span>;</div><div class="line"> <span class="comment">// Copy one whole row:</span></div><div class="line"> memcpy(dst_data + <span class="keywordtype">id</span>.z() * (width * height) + <span class="keywordtype">id</span>.y() * width, output_it.ptr(), width * <span class="keyword">sizeof</span>(float));</div><div class="line"> },</div><div class="line"> output_it);</div><div class="line"></div></div><!-- fragment --> <h2><a class="anchor" id="S4_6_5_sub_tensors"></a> |
| Sub-tensors</h2> |
| <p>Sub-tensors are aliases to existing Tensors, as a result creating a sub-tensor does not result in any underlying memory allocation.</p> |
| <p>Sub-tensors can be used to access a sub-set of the parent tensor, something that can be useful in case different operations need to be performed on different parts of a tensor.</p> |
| <p>Moreover, sub-tensors can be used to perform zero copy tensor concatenation.</p> |
| <p>The API for creating a sub-tensor is the following: </p><div class="fragment"><div class="line">SubTensor(ITensor *parent, <span class="keyword">const</span> TensorShape &tensor_shape, <span class="keyword">const</span> Coordinates &coords)</div></div><!-- fragment --><p>Where <em>parent</em> is the parent tensor which we want to create an alias for, <em>tensor_shape</em> is the shape of the sub-tensor and <em>coords</em> are the starting indexing coordinates of the sub-tensor within the parent tensor.</p> |
| <dl class="section note"><dt>Note</dt><dd>Two sub-tensor concrete classes for different targets are currently supported : <a class="el" href="classarm__compute_1_1_c_l_sub_tensor.xhtml">CLSubTensor</a> and <a class="el" href="classarm__compute_1_1_sub_tensor.xhtml">SubTensor</a></dd></dl> |
| <dl class="section warning"><dt>Warning</dt><dd>Limitation of the sub-tensor is that it cannot be extracted spatially, meaning sub-tensors should have the same width and height as the parent tensor. The main reasons for this is the fact that individual kernels might need to operate with a step size that is not a multiple of the sub-tensor spatial dimension. This could lead to elements being overwritten by different kernels operating on different sub-tensors of the same underlying tensor.</dd></dl> |
| <h1><a class="anchor" id="S4_7_memory_manager"></a> |
| MemoryManager</h1> |
| <p><a class="el" href="classarm__compute_1_1_i_memory_manager.xhtml">IMemoryManager</a> is a memory managing interface that can be used to reduce the memory requirements of a given pipeline by recycling temporary buffers.</p> |
| <h2><a class="anchor" id="S4_7_1_memory_manager_components"></a> |
| MemoryGroup, MemoryPool and MemoryManager Components</h2> |
| <h3><a class="anchor" id="S4_7_1_1_memory_group"></a> |
| MemoryGroup</h3> |
| <p><a class="el" href="classarm__compute_1_1_i_memory_group.xhtml">IMemoryGroup</a> defines the memory managing granularity.</p> |
| <p><a class="el" href="classarm__compute_1_1_memory_group.xhtml" title="Memory group.">MemoryGroup</a> binds a number of objects to a bucket of memory requirements that need to be fulfilled in order for an operation or list of operations to be executed.</p> |
| <p>Requesting backing memory for a specific group can be done using <a class="el" href="classarm__compute_1_1_i_memory_group.xhtml#accdbc89f54447d25dfc018d38070f4ea">IMemoryGroup::acquire</a> and releasing the memory back using <a class="el" href="classarm__compute_1_1_i_memory_group.xhtml#aab0a52fdd148a54108e7bf49287d7c47">IMemoryGroup::release</a>.</p> |
| <h3><a class="anchor" id="S4_7_1_2_memory_pool"></a> |
| MemoryPool</h3> |
| <p><a class="el" href="classarm__compute_1_1_i_memory_pool.xhtml">IMemoryPool</a> defines a pool of memory that can be used to provide backing memory to a memory group.</p> |
| <dl class="section note"><dt>Note</dt><dd><a class="el" href="classarm__compute_1_1_blob_memory_pool.xhtml">BlobMemoryPool</a> is currently implemented which models the memory requirements as a vector of distinct memory blobs.</dd></dl> |
| <h3><a class="anchor" id="S4_7_1_2_memory_manager_components"></a> |
| MemoryManager Components</h3> |
| <p><a class="el" href="classarm__compute_1_1_i_memory_manager.xhtml">IMemoryManager</a> consists of two components:</p><ul> |
| <li><a class="el" href="classarm__compute_1_1_i_lifetime_manager.xhtml">ILifetimeManager</a> that keeps track of the lifetime of the registered objects of the memory groups and given an <a class="el" href="classarm__compute_1_1_i_allocator.xhtml">IAllocator</a> creates an appropriate memory pool that fulfils the memory requirements of all the registered memory groups.</li> |
| <li><a class="el" href="classarm__compute_1_1_i_pool_manager.xhtml">IPoolManager</a> that safely manages the registered memory pools.</li> |
| </ul> |
| <dl class="section note"><dt>Note</dt><dd><a class="el" href="classarm__compute_1_1_blob_lifetime_manager.xhtml">BlobLifetimeManager</a> is currently implemented which models the memory requirements as a vector of distinct memory blobs.</dd></dl> |
| <h2><a class="anchor" id="S4_7_2_working_with_memory_manager"></a> |
| Working with the Memory Manager</h2> |
| <p>Using a memory manager to reduce the memory requirements of a pipeline can be summed in the following steps:</p> |
| <p>Initially a memory manager must be set-up: </p><div class="fragment"><div class="line">Allocator <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a3d45498be87235b5165d9972b2d0644d">allocator</a>{}; <span class="comment">// Create an allocator to use for the backing memory allocation</span></div><div class="line"><span class="keyword">auto</span> lifetime_mgr = std::make_shared<BlobLifetimeManager>(); <span class="comment">// Create Lifetime Manager</span></div><div class="line"><span class="keyword">auto</span> pool_mgr = std::make_shared<PoolManager>(); <span class="comment">// Create Pool Manager</span></div><div class="line"><span class="keyword">auto</span> mm = std::make_shared<MemoryManagerOnDemand>(lifetime_mgr, pool_mgr); <span class="comment">// Create Memory Manager</span></div></div><!-- fragment --><p>Once done, memory groups can be registered to use the memory manager: </p><div class="fragment"><div class="line">MemoryGroup memory_group(mm); <span class="comment">// Create a memory group and set the memory manager to use</span></div></div><!-- fragment --><dl class="section note"><dt>Note</dt><dd>If a memory manager is not specified then all allocation will be immediate instead of deferred through the memory manager.</dd></dl> |
| <p>Next step is to set objects to be managed by the memory group. It is important though to note that the lifetime of an object is tracked from the <a class="el" href="classarm__compute_1_1_memory_group.xhtml#a6fc0a49304c152c20a0f6df0634fb3cd">MemoryGroup::manage()</a> and the <a class="el" href="classarm__compute_1_1_tensor_allocator.xhtml#a6e509c2a177b0b29e9e2369535094dee">TensorAllocator::allocate</a> calls. <a class="el" href="classarm__compute_1_1_memory_group.xhtml#a6fc0a49304c152c20a0f6df0634fb3cd">MemoryGroup::manage</a> flags that the object will be needed starting now and when <a class="el" href="classarm__compute_1_1_tensor_allocator.xhtml#a6e509c2a177b0b29e9e2369535094dee">TensorAllocator::allocate</a> is called it signals the end of the object lifetime. </p><div class="fragment"><div class="line">Tensor tmp1, tmp2, tmp3; <span class="comment">// Create example tensors</span></div><div class="line">memory_group.manage(&tmp1); <span class="comment">// Start managing object tmp1 and start its lifetime</span></div><div class="line">memory_group.manage(&tmp2); <span class="comment">// Start managing object tmp2 and start its lifetime</span></div><div class="line"></div><div class="line">operation1.configure(&tmp1, &tmp2); <span class="comment">// Configure a function/kernel using tmp1 and tmp2</span></div><div class="line"></div><div class="line">tmp1.allocator()->allocate(); <span class="comment">// Flag that the lifetime of object tmp1 has ended</span></div><div class="line"></div><div class="line">memory_group.manage(&tmp3); <span class="comment">// Start managing object tmp3 and start its lifetime</span></div><div class="line"></div><div class="line">operation2.configure(&tmp2, &tmp3); <span class="comment">// Configure a function/kernel using tmp2 and tmp3</span></div><div class="line"></div><div class="line">tmp2.allocator()->allocate(); <span class="comment">// Flag that the lifetime of object tmp2 has ended</span></div><div class="line">tmp3.allocator()->allocate(); <span class="comment">// Flag that the lifetime of object tmp3 has ended</span></div></div><!-- fragment --><dl class="section warning"><dt>Warning</dt><dd>The configuration step should be done sequentially by a single thread so that all the lifetimes are captured correclty.</dd></dl> |
| <p>When configuration of all the operations is finished then the memory manager have to be populated: </p><div class="fragment"><div class="line">mm->populate(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a3d45498be87235b5165d9972b2d0644d">allocator</a>), 2 <span class="comment">/* num_pools */</span>); <span class="comment">// Populate memory manager pools</span></div></div><!-- fragment --><p>Finally, during execution of the pipeline the memory of the appropriate memory group should be requested before running: </p><div class="fragment"><div class="line">memory_group.acquire(); <span class="comment">// Request memory for the group</span></div><div class="line"></div><div class="line">operation1.run(); <span class="comment">// Run operation1</span></div><div class="line">operation2.run(); <span class="comment">// Run operation2</span></div><div class="line"></div><div class="line">memory_group.release(); <span class="comment">// Release memory so that it can be reused</span></div></div><!-- fragment --> <dl class="section note"><dt>Note</dt><dd>Execution of a pipeline can be done in a multi-threading environment as memory acquisition/release are thread safe. </dd> |
| <dd> |
| If you are handling sensitive data and it's required to zero out the memory buffers before freeing, make sure to also zero out the intermediate buffers. You can access the buffers through the memory group's mappings.</dd></dl> |
| <h2><a class="anchor" id="S4_7_3_memory_manager_function_support"></a> |
| Function support</h2> |
| <p>Most of the library's function have been ported to use <a class="el" href="classarm__compute_1_1_i_memory_manager.xhtml">IMemoryManager</a> for their internal temporary buffers.</p> |
| <p>If that is the case, a memory manager can be passed to them during construction to reuse memory among these functions. </p><div class="fragment"><div class="line"><span class="comment">// Setup Memory Manager</span></div><div class="line">CLBufferAllocator <a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a3d45498be87235b5165d9972b2d0644d">allocator</a>{}; <span class="comment">// Create an allocator to use for the backing memory allocation</span></div><div class="line"><span class="keyword">auto</span> lifetime_mgr = std::make_shared<BlobLifetimeManager>(); <span class="comment">// Create Lifetime Manager</span></div><div class="line"><span class="keyword">auto</span> pool_mgr = std::make_shared<PoolManager>(); <span class="comment">// Create Pool Manager</span></div><div class="line"><span class="keyword">auto</span> mm = std::make_shared<MemoryManagerOnDemand>(lifetime_mgr, pool_mgr); <span class="comment">// Create Memory Manager</span></div><div class="line"></div><div class="line"><span class="comment">// Create two convolution layers and use the memory manager to manager their internal temporary buffers</span></div><div class="line">CLConvolutionLayer conv1(mm), conv2(mm);</div><div class="line"></div><div class="line"><span class="comment">// Configure layers</span></div><div class="line">conv1.configure(...);</div><div class="line">conv2.configure(...);</div><div class="line"></div><div class="line"><span class="comment">// Populate memory manager</span></div><div class="line">mm->populate(&<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a3d45498be87235b5165d9972b2d0644d">allocator</a>), 1 <span class="comment">/* num_pools */</span>); <span class="comment">// Populate memory manager pools</span></div><div class="line"></div><div class="line"><span class="comment">// Run layers (Memory will be recycled for internal buffers for conv1 and conv2</span></div><div class="line">conv1.run();</div><div class="line">conv2.run();</div></div><!-- fragment --><h1><a class="anchor" id="S4_8_import_memory"></a> |
| Import Memory Interface</h1> |
| <p>The implemented <a class="el" href="classarm__compute_1_1_tensor_allocator.xhtml">TensorAllocator</a> and <a class="el" href="classarm__compute_1_1_c_l_tensor_allocator.xhtml">CLTensorAllocator</a> objects provide an interface capable of importing existing memory to a tensor as backing memory.</p> |
| <p>A simple NEON example can be the following: </p><div class="fragment"><div class="line"><span class="comment">// External backing memory</span></div><div class="line"><span class="keywordtype">void</span>* external_ptr = ...;</div><div class="line"></div><div class="line"><span class="comment">// Create and initialize tensor</span></div><div class="line">Tensor tensor;</div><div class="line">tensor.allocator()->init(tensor_info);</div><div class="line"></div><div class="line"><span class="comment">// Import existing pointer as backing memory</span></div><div class="line">tensor.allocator()->import_memory(external_ptr);</div></div><!-- fragment --><p>It is important to note the following:</p><ul> |
| <li>Ownership of the backing memory is not transferred to the tensor itself.</li> |
| <li>The tensor mustn't be memory managed.</li> |
| <li>Padding requirements should be accounted by the client code. In other words, if padding is required by the tensor after the function configuration step, then the imported backing memory should account for it. Padding can be checked through the <a class="el" href="classarm__compute_1_1_tensor_info.xhtml#a79e20eacb1e963e24a21ebd7369effd7">TensorInfo::padding()</a> interface.</li> |
| </ul> |
| <h1><a class="anchor" id="S4_9_opencl_tuner"></a> |
| OpenCL Tuner</h1> |
| <p>OpenCL kernels when dispatched to the GPU take two arguments:</p><ul> |
| <li>The Global Workgroup Size (GWS): That's the number of times to run an OpenCL kernel to process all the elements we want to process.</li> |
| <li>The Local Workgroup Size (LWS): That's the number of elements we want to run in parallel on a GPU core at a given point in time.</li> |
| </ul> |
| <p>The LWS can be required by an algorithm (For example if it contains memory barriers or uses local memory) but it can also be used for performance reasons to tweak the performance of a kernel: the execution time of the overall kernel might vary significantly depending on how the GWS is broken down.</p> |
| <p>However, there is no universal rule regarding which LWS is best for a given kernel, so instead we created the <a class="el" href="classarm__compute_1_1_c_l_tuner.xhtml">CLTuner</a>.</p> |
| <p>When the <a class="el" href="classarm__compute_1_1_c_l_tuner.xhtml">CLTuner</a> is enabled ( Target = 2 for the graph examples), the first time an OpenCL kernel is executed the Compute Library will try to run it with a variety of LWS values and will remember which one performed best for subsequent runs. At the end of the run the <a class="el" href="classarm__compute_1_1graph_1_1_graph.xhtml">graph::Graph</a> will try to save these tuning parameters to a file.</p> |
| <p>However this process takes quite a lot of time, which is why it cannot be enabled all the time. <a class="el" href="classarm__compute_1_1_c_l_tuner.xhtml">CLTuner</a> supports three modes of tuning with different trade-offs between the time taken to tune and the kernel execution time achieved using the best LWS found. In the Exhaustive mode, it searches all the supported values of LWS. This mode takes the longest time to tune and is the most likely to find the optimal LWS. Normal mode searches a subset of LWS values to yield a good approximation of the optimal LWS. It takes less time to tune than Exhaustive mode. Rapid mode takes the shortest time to tune and finds an LWS value that is at least as good or better than the default LWS value. The mode affects only the search for the optimal LWS and has no effect when the LWS value is imported from a file.</p> |
| <p>But, when the <a class="el" href="classarm__compute_1_1_c_l_tuner.xhtml">CLTuner</a> is disabled ( Target = 1 for the graph examples), the <a class="el" href="classarm__compute_1_1graph_1_1_graph.xhtml">graph::Graph</a> will try to reload the file containing the tuning parameters, then for each executed kernel the Compute Library will use the fine tuned LWS if it was present in the file or use a default LWS value if it's not.</p> |
| <h1><a class="anchor" id="S4_10_weights_manager"></a> |
| Weights Manager</h1> |
| <p><a class="el" href="classarm__compute_1_1_i_weights_manager.xhtml">IWeightsManager</a> is a weights managing interface that can be used to reduce the memory requirements of a given pipeline by reusing transformed weights across multiple function executions. <a class="el" href="classarm__compute_1_1_i_weights_manager.xhtml">IWeightsManager</a> is responsible for managing weight tensors alongside with their transformations. <a class="el" href="classarm__compute_1_1_i_transform_weights.xhtml">ITransformWeights</a> provides an interface for running the desired transform function. This interface is used by the weights manager.</p> |
| <h2><a class="anchor" id="S4_10_1_working_with_weights_manager"></a> |
| Working with the Weights Manager</h2> |
| <p>Following is a simple example that uses the weights manager:</p> |
| <p>Initially a weights manager must be set-up: </p><div class="fragment"><div class="line"><span class="keyword">auto</span> wm = std::make_shared<IWeightsManager>(); <span class="comment">// Create a weights manager</span></div></div><!-- fragment --><p>Once done, weights can be managed, configured and run: </p><div class="fragment"><div class="line">wm->manage(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a64a08a9fec5aeee8650e7182b6d171d0">weights</a>); <span class="comment">// Manage the weights</span></div><div class="line">wm->acquire(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a64a08a9fec5aeee8650e7182b6d171d0">weights</a>, &_reshape_weights_managed_function); <span class="comment">// Acquire the address of the transformed weights based on the transform function</span></div><div class="line">wm->run(<a class="code" href="namespacearm__compute_1_1test_1_1validation.xhtml#a64a08a9fec5aeee8650e7182b6d171d0">weights</a>, &_reshape_weights_managed_function); <span class="comment">// Run the transpose function</span></div></div><!-- fragment --><h1><a class="anchor" id="S5_0_experimental"></a> |
| Experimental Features</h1> |
| <h2><a class="anchor" id="S5_1_run_time_context"></a> |
| Run-time Context</h2> |
| <p>Some of the Compute Library components are modelled as singletons thus posing limitations to supporting some use-cases and ensuring a more client-controlled API. Thus, we are introducing an aggregate service interface <a class="el" href="classarm__compute_1_1_i_runtime_context.xhtml">IRuntimeContext</a> which will encapsulate the services that the singletons were providing and allow better control of these by the client code. Run-time context encapsulates a list of mechanisms, some of them are: scheduling, memory management, kernel caching and others. Consequently, this will allow better control of these services among pipelines when Compute Library is integrated in higher level frameworks.</p> |
| <p>This feature introduces some changes to our API. All the kernels/functions will now accept a Runtime Context object which will allow the function to use the mentioned services. Moreover, all the objects will require to be created using the context to have access to these services. Note that these will apply to the runtime components as the core ones do not need access to such services. The only exception is the kernel caching mechanism which will need to be passed down at kernel level.</p> |
| <p>Finally, we will try to adapt our code-base progressively to use the new mechanism but will continue supporting the legacy mechanism to allow a smooth transition. Changes will apply to all our three backends: NEON, OpenCL and OpenGL ES. </p> |
| </div></div><!-- PageDoc --> |
| </div><!-- contents --> |
| </div><!-- doc-content --> |
| <!-- start footer part --> |
| <div id="nav-path" class="navpath"><!-- id is needed for treeview function! --> |
| <ul> |
| <li class="footer">Generated on Thu Mar 5 2020 16:07:08 for Compute Library by |
| <a href="http://www.doxygen.org/index.html"> |
| <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.15 </li> |
| </ul> |
| </div> |
| </body> |
| </html> |
