ipynb/releases/ReleaseNotes_v17.03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Documentation of many LISA modules has got a big improvement with the usage of Sphinx and the refresh of docstrings for many existing methods.\n",
    "\n",
    "You can access documentation either interactively in Notebooks, using the standard TAB completion after a function name, or by printing it in the Notebook itsels. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "run_control": {
     "marked": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "    A base class for LISA tests\n",
      "\n",
      "    This class is intended to be subclassed in order to create automated tests\n",
      "    for LISA. It sets up the TestEnv and Executor and provides convenience\n",
      "    methods for making assertions on results.\n",
      "\n",
      "    Subclasses should provide a test_conf to configure the TestEnv and an\n",
      "    experiments_conf to configure the executor.\n",
      "\n",
      "    Tests whose behaviour is dependent on target parameters, for example\n",
      "    presence of cpufreq governors or number of CPUs, can override\n",
      "    _getExperimentsConf to generate target-dependent experiments.\n",
      "\n",
      "    Example users of this class can be found under LISA's tests/ directory.\n",
      "\n",
      "    :ivar experiments: List of :class:`Experiment` s executed for the test. Only\n",
      "                       available after :meth:`init` has been called.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from test import LisaTest\n",
    "\n",
    "print LisaTest.__doc__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Energy Model Related APIs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `EnergyModel` class has been added, which provides methods for describing platforms in order to estimate usage of CPU systems under various utilization scenario. The model is aware of frequency (DVFS) domains, power domains and idle states, as well as \"cluster\" energy.\n",
    "\n",
    "Tests have been added that utulize the `EnergyModel` - see below for info about the Generic tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Represents hierarchical CPU topology with power and capacity data\n",
      "\n",
      "    An energy model consists of\n",
      "\n",
      "    - A CPU topology, representing the physical (cache/interconnect) topology of\n",
      "      the CPUs.  Each node stores the energy usage of that node's hardware when\n",
      "      it is in each active or idle state. They also store a compute capacity at\n",
      "      each frequency, but this is only meaningful for leaf nodes (CPUs) and may\n",
      "      be None at higher levels. These capacity values are relative; the maximum\n",
      "      capacity would usually be 1024, the value of SCHED_CAPACITY_SCALE in the\n",
      "      Linux kernel scheduler.  Use EnergyModelNodes to describe this.\n",
      "\n",
      "    - A power domain topology, representing the hierarchy of areas that can be\n",
      "      powered down (idled).\n",
      "      The power domains are a single tree. Leaf nodes must contain exactly one\n",
      "      CPU and the root node must indirectly contain every CPU. Each power domain\n",
      "      has a list (maybe empty) of names of idle states that that domain can\n",
      "      enter.\n",
      "      Use PowerDomains to describe this.\n",
      "\n",
      "    - A set of frequency domains, representing groups of CPUs whose clock\n",
      "      frequencies must be equal (probably because they share a clock). The\n",
      "      frequency domains must be a partition of the CPUs.\n",
      "\n",
      "    :ivar cpu_nodes: List of leaf (CPU) :class:`EnergyModelNode`\n",
      "    :ivar cpus: List of logical CPU numbers in the system\n",
      "\n",
      "    :param root_node: Root of :class:`EnergyModelNode` tree\n",
      "    :param root_power_domain: Root of :class:`PowerDomain` tree\n",
      "    :param freq_domains: Collection of collections of logical CPU numbers\n",
      "                         representing frequency (clock) domains.\n",
      "\n",
      "    .. note::\n",
      "      The most signficant shortcomings of the model are:\n",
      "\n",
      "        1. Voltage domains are assumed to be congruent to frequency domains\n",
      "\n",
      "        2. Idle state power is assumed to be independent of voltage\n",
      "\n",
      "        3. Temperature is ignored entirely\n",
      "\n",
      "    .. _cpu-utils:\n",
      "\n",
      "    .. admonition:: ``cpu_utils``: CPU util distributions\n",
      "\n",
      "        Used throughout this module: A ``cpu_utils`` is a list ``u`` where\n",
      "        ``u[N]`` is the sum of the frequency-invariant, capacity-invariant\n",
      "        utilization of tasks placed on CPU N. That is, the quantity represented\n",
      "        by a CPU runqueue's util_avg in the Linux kernel scheduler's\n",
      "        load-tracking system with EAS features enabled.\n",
      "\n",
      "        The range of utilization values is 0 -\n",
      "        :attr:`EnergyModel.capacity_scale`.\n",
      "\n",
      "        This represents a static utilization, assuming that tasks don't change\n",
      "        in size (for example representing a set of fixed periodic RT-App\n",
      "        workloads). For workloads that change over time, a series of\n",
      "        ``cpu_utils`` items would be needed to describe the utilization, with a\n",
      "        distinct estimation for each item in the series.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from energy_model import EnergyModel\n",
    "print EnergyModel.__doc__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7AAAACZCAYAAAAW9tahAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADyRJREFUeJzt3XmsrHV5B/Dvg7hRQaEQRBCvVaqlraIimkoqaq0YrWhK\nrTZV2mj5R1NMG+L2h/aPppqu/5g0JC5oW0mrteBeRKtBjSxKRcUWqYBYBFwquKCiT/84L3ru4Sxz\nl5l533M+n2QyM7/3zHmeO+d+773PnXep7g4AAACM3QHLbgAAAABmYYAFAABgEgywAAAATIIBFgAA\ngEkwwAIAADAJBlgAAAAmwQALAADAJBhgAQAAmAQDLAAAAJNw4LIbmMXhhx/eu3btWnYbsFSXX375\n17v7iGX3sR4ZBRmFsZNRGLdZMzqJAXbXrl257LLLlt0GLFVVXbfsHjYioyCjMHYyCuM2a0btQgwA\nAMAkGGABAACYBAMsAAAAkzCJY2AZv12veO8+vf7a1z1jP3UCsD5/TsHG5GPc/HzgZ3wCCwAAwCQY\nYAEAAJgEAywAAACTYIAFAABgEgywAAAATIIBFgAAgEkwwAIAADAJBlgAAAAmwQALAADAJBhgAQAA\nmAQDLAAAAJNggAUAAGASDLAAAABMggEWAACASTDAAgAAMAkGWAAAACbBAAsAAMAkzG2AraoHVtVH\nquoLVfX5qjprWD+sqi6sqquH+0Pn1QMAAADbxzw/gb0jyZ929/FJHp/kJVV1fJJXJLmou49LctHw\nHAAAADY1twG2u2/s7k8Pj29LclWSo5OcluTc4cvOTfLsefUAAADA9rGQY2CraleSRyX5VJIju/vG\nYdPXkhy5wWvOHHYzvuX6669fRJvAHpBRGDcZhXGTUdg7cx9gq+o+Sd6Z5GXdfevqbd3dSXq913X3\nOd19XHcfceyxx867TWAPySiMm4zCuMko7J25DrBVdfesDK//2N3/OizfVFVHDduPSnLzPHsAAABg\ne5jnWYgryRuTXNXdf7Nq0wVJzhgen5Hk/Hn1AAAAwPZx4By/9xOSvCDJlVV1xbD2qiSvS/LPVfWi\nJNclee4cewAAAGCbmNsA290XJ6kNNj9lXnUBAADYnmYeYKvq15LsWv2a7n7rHHoCAACAu5hpgK2q\ntyV5SJIrkvx4WO4kBlgAAAAWYtZPYE9Mcvxw2RsAAABYuFnPQvy5JPefZyMAAACwmVk/gT08yReq\n6pIkP7hzsbufNZeuAAAAYI1ZB9jXzrMJAAAA2MpMA2x3f7Sqjkzy2GHpku6+eX5tAQAAwO5mOga2\nqp6b5JIkv5PkuUk+VVWnz7MxAAAAWG3WXYhfneSxd37qWlVHJPlQknfMqzEAAABYbdazEB+wZpfh\nb+zBawEAAGCfzfoJ7Aeq6oNJ3j48/90k75tPSwAAAHBXs57E6eyq+u0kTxiWzunud82vLQAAANjd\nrJ/AprvfmeSdc+wFAAAANrTpAFtVF3f3yVV1W5JevSlJd/chc+0OAAAABpsOsN198nB/8GLaAQAA\ngPXNeh3Yt82yBgAAAPMy66Vwfnn1k6o6MMlj9n87AAAAsL5NB9iqeuVw/OsjqurW4XZbkpuSnL+Q\nDgEAACBbDLDd/RfD8a9/2d2HDLeDu/vnu/uVC+oRAAAAZr4O7Cur6tAkxyW516r1j82rMQAAAFht\npgG2ql6c5KwkxyS5Isnjk3wyyZPn1xoAAAD8zKwncToryWOTXNfdT0ryqCT/N7euAAAAYI1ZB9jb\nu/v2JKmqe3b3F5M8bH5tAQAAwO5m2oU4yQ1Vdb8k/5bkwqr6VpLr5tcWAAAA7G7Wkzg9Z3j42qr6\nSJL7Jnn/3LoCAACANWbahbiq3lhVJyRJd3+0uy9I8uq5dgYAAACrzHoM7NOSnFtVZ6xae9Yc+gEA\nAIB1zTrA3pzk15OcXlVvqKoDk9T82gIAAIDdzTrAVnd/u7t/K8ktSf4jK8fBAgAAwELMOsBecOeD\n7n5tktcnuXYO/QAAAMC6Zhpgu/s1a56/u7ufPJ+WAAAA4K42vYxOVV3c3SdX1W1JevWmJN3dh8y1\nOwAAABhs+glsd5883B/c3Yesuh281fBaVW+qqpur6nOr1g6rqgur6urh/tD988sAAABgu9t0gB0G\nzg1vW3zvtyQ5dc3aK5Jc1N3HJbloeA4AAABb2nQX4iSXZ2XX4fUumdNJfmGjF3b3x6pq15rl05Kc\nMjw+NytnM3751m0CAACw0206wHb3g/dzvSO7+8bh8deSHLnRF1bVmUnOTnK/I444Yj+3AewrGYVx\nk1EYNxmFvbPVLsQPH+4fvd5tXwp3d2f3E0Ot3X5Odx/X3Ucce+yx+1IKmAMZhXGTURg3GYW9s9Uu\nxH+S5Mwkf73Otk6yp5fSuamqjuruG6vqqCQ37+HrAQAA2KG22oX4zOHh07v79tXbqupee1HvgiRn\nJHndcH/+XnwPAAAAdqBNdyFe5RMzrv1UVb09ySeTPKyqbqiqF2VlcH1qVV2d5DeG5wAAALClTT+B\nrar7Jzk6yb3XHPN6SJKDNnttdz9/g01P2aMOAQAAIFsfA/u0JH+Q5Jgkf7Vq/bYkr5pTTwAAAHAX\nWw2whyd5z3BLVk7cdEuSi7v7y/NsDAAAAFbb6hjY+6y5HZzkxCTvr6rnzbk3AAAA+KmtzkL8Z+ut\nV9VhST6U5Lx5NAUAAABrzXoW4t109zeT1H7uBQAAADa0VwNsVT0pybf2cy8AAACwoa0uo3NlVk7c\ntNphSf43yQvn1RQAAACstdVZiJ+55nkn+UZ3f3dO/QAAAMC6tjqJ03WLagQAAAA2s1fHwAIAAMCi\nGWABAACYBAMsAAAAk2CABQAAYBIMsAAAAEyCARYAAIBJMMACAAAwCQZYAAAAJsEACwAAwCQYYAEA\nAJgEAywAAACTYIAFAABgEgywAAAATIIBFgAAgEkwwAIAADAJBlgAAAAmwQALAADAJBhgAQAAmAQD\nLAAAAJNggAUAAGASDLAAAABMggEWAACASTDAAgAAMAlLGWCr6tSq+q+q+lJVvWIZPQAAADAtCx9g\nq+puSd6Q5OlJjk/y/Ko6ftF9AAAAMC3L+AT2pCRf6u7/6e4fJjkvyWlL6AMAAIAJOXAJNY9O8pVV\nz29I8ri1X1RVZyY5O8n9ktxeVZ/fh5qHJ/n6Prx+Xy27/hh62LR+vX659RdkX3t40P5qZH+Q0W3X\nw06vv2UPM/w5JaPzs+z6Y+hh1PUX8Pf4lj3MYMdmdAf8O2vZ9cfQw3aoP1NGq7v3sc6eqarTk5za\n3S8enr8gyeO6+6VzrHlZd584r+8/9vpj6GGn1x9LD2O17Pdm2fXH0MNOrz+WHsZq2e/NsuuPoYed\nXn8sPYzVst+bnV5/DD3spPrL2IX4q0keuOr5McMaAAAAbGgZA+ylSY6rqgdX1T2SPC/JBUvoAwAA\ngAlZ+DGw3X1HVb00yQeT3C3Jm7p7X/b5n8U5c/7+Y6+fLL+HnV4/GUcPY7Xs92bZ9ZPl97DT6yfj\n6GGslv3eLLt+svwednr9ZBw9jNWy35udXj9Zfg87pv7Cj4EFAACAvbGMXYgBAABgjxlgAQAAmAQD\nLAAAAJNggAUAAGASFn4W4kWoqocnOS3J0cPSV5Nc0N1XLa+rxRl+/Ucn+VR3f2fV+qnd/YEF9XBS\nku7uS6vq+CSnJvlid79vEfXX6eet3f3CZdQe6p+c5KQkn+vuf19WH2MhozK6Tj9Ly6h87m6n5zOR\n0Q36kdGRkNHlZ3Rs+Rx62jEZ3XZnIa6qlyd5fpLzktwwLB+TlevNntfdr1tib3/Y3W+ec40/TvKS\nJFclOSHJWd19/rDt09396HnWH+q8JsnTs/IfJBcmeVySjyR5apIPdvefz7n+2usKV5InJflwknT3\ns+ZZf+jhku4+aXj8R1n5mbwryW8mefcyfx8um4zK6LIzKp8bG3M+ExmNjMqojC49o8vO59DDzs5o\nd2+rW5L/TnL3ddbvkeTqJfd2/QJqXJnkPsPjXUkuy0qwk+QzC/p1XpmVa/welOTWJIcM6/dO8tkF\n1P90kn9IckqSJw73Nw6Pn7ig9+Azqx5fmuSI4fHPJblyET2M9SajMrrsjMrnpu/NaPM59CGjMiqj\nMrrUjC47n0OtHZ3R7bgL8U+SPCDJdWvWjxq2zVVVfXajTUmOnHf9JAf0sCtFd19bVackeUdVPWjo\nYRHu6O4fJ/leVV3T3bcO/Xy/qub+M0hyYpKzkrw6ydndfUVVfb+7P7qA2nc6oKoOzcpx5tXdtyRJ\nd3+3qu5YYB9jJKMyuuyMyufGlprPREYHMiqjG5HR5Wd02flMdnhGt+MA+7IkF1XV1Um+Mqwdm+Sh\nSV66gPpHJnlakm+tWa8kn1hA/Zuq6oTuviJJuvs7VfXMJG9K8qsLqJ8kP6yqg7r7e0kec+diVd03\nC/jDtbt/kuRvq+pfhvubsvjf6/dNcnlWfu5dVUd1941VdZ8s7h9AYyWjMrrsjMrnxpadz0RGExmV\n0Y3J6PIzutR8JjK67Y6BTZKqOiArBxKvPrj90uF/S+Zd+41J3tzdF6+z7Z+6+/fmXP+YrPzP0NfW\n2faE7v74POsPde7Z3T9YZ/3wJEd195Xz7mFN3WckeUJ3v2qRdTfo5aAkR3b3l5fdyzLJqIyuqTuK\njMrnimXmc6gvozK6UR8yGhlddkbHls+h9o7K6LYcYAEAANh+XAcWAACASTDAAgAAMAkGWFJV96+q\n86rqmqq6vKreV1W/WFXfr6orquoLVfX3VXVAVZ1SVe9Z8/q3VNXpy+oftjsZhXGTURg3Gd1etuNZ\niNkDVVVZufDwud39vGHtkVk5w9w13X1CVR2YlQsjPzvJN5fWLOxAMgrjJqMwbjK6/fgElicl+VF3\n//2dC939n/nZqdnT3Xdk5bToD118e7DjySiMm4zCuMnoNmOA5Veych2nDQ2nxH5KkoWfFhyQURg5\nGYVxk9FtxgDLZh5SVVck+XiS93b3+5NsdN0l12OCxZNRGDcZhXGT0QlyDCyfT7LRQenXdPcJa9a+\nkeTQNWuHJfn6/m4MSCKjMHYyCuMmo9uMT2D5cJJ7VtWZdy5U1SOSPHCDr786yQOq6peGr31Qkkcm\nuWLejcIOJaMwbjIK4yaj24xPYHe47u6qek6Sv6uqlye5Pcm1SV62wdf/oKp+P8mbq+peSX6U5MXd\n/e1F9Qw7iYzCuMkojJuMbj/VbXduAAAAxs8uxAAAAEyCARYAAIBJMMACAAAwCQZYAAAAJsEACwAA\nwCQYYAEAAJgEAywAAACT8P/SBf9A+hAXEAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9b5e3a0050>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# juno_energy provides an instance of EnergyModel for ARM Juno platforms\n",
    "from platforms.juno_energy import juno_energy\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "possible_placements = juno_energy.get_optimal_placements({'task1': 10, 'task2': 15})\n",
    "fig, axs = plt.subplots(1, 4, sharey=True)\n",
    "# fig.set_ylabel('Utilization')\n",
    "for ax, placement in zip(axs, possible_placements):\n",
    "    ax.set_ylabel('Utilization')\n",
    "    ax.set_xlabel('CPU')\n",
    "\n",
    "    pd.DataFrame(list(placement)).plot(kind='bar', figsize=(16, 2), ax=ax, legend=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above example shows how the `EnergyModel` class can be used to find optimal task placements. Here it is shown that on ARM Juno, if the system is presented with just two small tasks, it should place them on the same CPU, not using the big CPUs (1 and 2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Trace module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improved profiling analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The trace analysis module has got a more complete support for analysis of tasks properties.\n",
    "\n",
    "Here is an example notebook which shows the new API in use on a relatively simple example:\n",
    "\n",
    "https://gist.github.com/derkling/256256f47bc9daf4883f3cb6e356e26b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Android Support"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API to run Android Workloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new API has been adde which allows to defined how to execute and Android workload with the additional support:\n",
    "- to collect a trace across its execution\n",
    "- to measure energy consumption across its execution "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "run_control": {
     "marked": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "    Base class for Android related workloads\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from android import Workload\n",
    "\n",
    "print Workload.__doc__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Public interface:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "run_control": {
     "marked": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'getInstance': <classmethod at 0x7f9b5e1c5398>,\n",
       " 'run': <function android.workload.run>,\n",
       " 'tracingStart': <function android.workload.tracingStart>,\n",
       " 'tracingStop': <function android.workload.tracingStop>}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{ k:v for k,v in vars(Workload).items() if not k.startswith('_') }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `run` method is the only one which the user is required to implement to specify how to run the specific Android workload.\n",
    "\n",
    "\n",
    "To create a new workload it's required to create a new module under this folder:\n",
    "`libs/utils/android/workloads`\n",
    "\n",
    "Here is an enample of usage of this class to run a YouTube workload:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/libs/utils/android/workloads/youtube.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Android Workloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `Workload` class, some interesting Android workloads have been already integrated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;34m/home/brendan/sources/lisa/libs/utils/android/workloads\u001b[00m\r\n",
      "├── geekbench.py\r\n",
      "├── geekbench.pyc\r\n",
      "├── gmaps.py\r\n",
      "├── gmaps.pyc\r\n",
      "├── jankbench.py\r\n",
      "├── jankbench.pyc\r\n",
      "├── \u001b[01;32muibench.py\u001b[00m\r\n",
      "├── uibench.pyc\r\n",
      "├── vellamo.py\r\n",
      "├── vellamo.pyc\r\n",
      "├── youtube.py\r\n",
      "└── youtube.pyc\r\n",
      "\r\n",
      "0 directories, 12 files\r\n"
     ]
    }
   ],
   "source": [
    "!tree $LISA_HOME/libs/utils/android/workloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and others are on their way ;-)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "## API to run Android Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new API has been added which allows to defined how to run an Android workload to perform a pre-defined set of experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "    A base class for LISA custom benchmarks execution\n",
      "\n",
      "    This class is intended to be subclassed in order to create a custom\n",
      "    benckmark execution for LISA.\n",
      "    It sets up the TestEnv and and provides convenience methods for\n",
      "    test environment setup, execution and post-processing.\n",
      "\n",
      "    Subclasses should provide a bm_conf to setup the TestEnv and\n",
      "    a set of optional callback methods to configuere a test environment\n",
      "    and process collected data.\n",
      "\n",
      "    Example users of this class can be found under LISA's tests/benchmarks\n",
      "    directory.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from android import LisaBenchmark\n",
    "\n",
    "print LisaBenchmark.__doc__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Public interface:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'benchmarkFinalize': <function android.benchmark.benchmarkFinalize>,\n",
       " 'benchmarkInit': <function android.benchmark.benchmarkInit>,\n",
       " 'bm_collect': None,\n",
       " 'bm_conf': None,\n",
       " 'bm_name': None,\n",
       " 'bm_params': None,\n",
       " 'reboot_target': <function android.benchmark.reboot_target>}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{ k:v for k,v in vars(LisaBenchmark).items() if not k.startswith('_') }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a test the used need to define proper values for the `bm_*` attributes.\n",
    "\n",
    "Optionally the `benchmarkInit` and `benchmarkFinalize` methods can also be defined.\n",
    "\n",
    "Here is an example usage of this API to run a YouTube workload using different CPUFreq governor:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/tests/benchmarks/android_youtube.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The new test API and LISA support allows to run tests both in batch, useful for regression testing, as well as interactively on a Notebook. This last mode is partitularely useful to support both tests development as well as to do further interactive analysis based on test results. For example it makes more easy to investigate the reasons for a test failure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "## Generic EAS tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://github.com/ARM-software/lisa/pull/263\n",
    "\n",
    "Provides a new full set of EAS behaviors testing based on EM data. These tests are designed to be completely generic and portable across different platform.\n",
    "\n",
    "The `Run Generic Automated EAS tests` example notebook:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/ipynb/tests/Generic_EAS_Tests.ipynb\n",
    "\n",
    "gives an example of using this test."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Frequency Invariant Test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://github.com/ARM-software/lisa/pull/254\n",
    "\n",
    "The frequency invariance machinary is a the base of a proper functioning of the EAS.\n",
    "We added a test which verifys frequency and cpu invarnace for load and utilization signals.\n",
    "\n",
    "The `Frequency Invariant Load Tracking Test` example notebook:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/ipynb/tests/Frequency_Invariance_Test.ipynb\n",
    "\n",
    "gives an example of using this test."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  },
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}