ipynb/releases/ReleaseNotes_v16.12.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Target Connectivity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configurable logging system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All LISA modules have been updated to use a more consistent logging which can be configured using a single configuraton file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "################################################################################\r\n",
      "### Root Logger\r\n",
      "################################################################################\r\n",
      "\r\n",
      "[loggers]\r\n",
      "keys=root\r\n",
      "\r\n",
      "[logger_root]\r\n",
      "level=INFO\r\n",
      "handlers=consoleHandler,fileHandler\r\n",
      "propagate=0\r\n"
     ]
    }
   ],
   "source": [
    "!head -n12 $LISA_HOME/logging.conf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each module has a unique name which can be used to assign a priority level for messages generated by that module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# [logger_TestEnv]\r\n",
      "# qualname=TestEnv\r\n",
      "# level=DEBUG\r\n",
      "# handlers=consoleHandler,fileHandler\r\n",
      "# propagate=0\r\n"
     ]
    }
   ],
   "source": [
    "!head -n30 $LISA_HOME/logging.conf | tail -n5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The default logging level for a notebook can also be easily configured using this few lines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "run_control": {
     "marked": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-12-15 17:15:08,209 INFO    : root         : Using LISA logging configuration:\n",
      "2016-12-15 17:15:08,209 INFO    : root         :   /home/patbel01/Code/lisa/logging.conf\n"
     ]
    }
   ],
   "source": [
    "import logging\n",
    "from conf import LisaLogging\n",
    "LisaLogging.setup(level=logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Removed Juno/Juno2 distinction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Juno R0 and Juno R2 boards are now accessible by specifying \"juno\" in the target configuration.\n",
    "The previous distinction was required because of a different way for the two boards to report HWMON channels.\n",
    "This distinction is not there anymore and thus Juno boards can now be connected using the same platform data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-12-15 16:49:03,716 INFO    : TestEnv      : Using base path: /home/patbel01/Code/lisa\n",
      "2016-12-15 16:49:03,716 INFO    : TestEnv      : Loading custom (inline) target configuration\n",
      "2016-12-15 16:49:03,717 INFO    : TestEnv      : Devlib modules to load: ['bl', 'hwmon', 'cpufreq']\n",
      "2016-12-15 16:49:03,717 INFO    : TestEnv      : Connecting linux target:\n",
      "2016-12-15 16:49:03,717 INFO    : TestEnv      :   username : root\n",
      "2016-12-15 16:49:03,718 INFO    : TestEnv      :       host : 10.1.210.45\n",
      "2016-12-15 16:49:03,718 INFO    : TestEnv      :   password : \n",
      "2016-12-15 16:49:03,718 INFO    : TestEnv      : Connection settings:\n",
      "2016-12-15 16:49:03,718 INFO    : TestEnv      :    {'username': 'root', 'host': '10.1.210.45', 'password': ''}\n",
      "2016-12-15 16:49:20,157 INFO    : TestEnv      : Initializing target workdir:\n",
      "2016-12-15 16:49:20,157 INFO    : TestEnv      :    /root/devlib-target\n",
      "2016-12-15 16:49:21,948 INFO    : TestEnv      : Topology:\n",
      "2016-12-15 16:49:21,949 INFO    : TestEnv      :    [[0, 3, 4, 5], [1, 2]]\n",
      "2016-12-15 16:49:23,594 INFO    : TestEnv      : Loading default EM:\n",
      "2016-12-15 16:49:23,594 INFO    : TestEnv      :    /home/patbel01/Code/lisa/libs/utils/platforms/juno.json\n",
      "2016-12-15 16:49:23,602 INFO    : EnergyMeter  : Scanning for HWMON channels, may take some time...\n",
      "2016-12-15 16:49:23,603 INFO    : EnergyMeter  : Channels selected for energy sampling:\n",
      "2016-12-15 16:49:23,603 INFO    : EnergyMeter  :    BOARDBIG_energy\n",
      "2016-12-15 16:49:23,604 INFO    : EnergyMeter  :    BOARDLITTLE_energy\n",
      "2016-12-15 16:49:23,604 INFO    : TestEnv      : Set results folder to:\n",
      "2016-12-15 16:49:23,604 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results/20161215_164923\n",
      "2016-12-15 16:49:23,604 INFO    : TestEnv      : Experiment results available also in:\n",
      "2016-12-15 16:49:23,605 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results_latest\n"
     ]
    }
   ],
   "source": [
    "from env import TestEnv\n",
    "\n",
    "te = TestEnv({\n",
    "        'platform'     : 'linux',\n",
    "        'board'        : 'juno',\n",
    "        'host'         : '10.1.210.45',\n",
    "        'username'     : 'root'\n",
    "    })\n",
    "target = te.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Executor Module\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simplified tests definition using in-code configurations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Automated LISA tests previously configured the Executor using JSON files. This is still possible, but the existing tests now use Python dictionaries directly in the code. In the short term, this allows de-duplicating configuration elements that are shared between multiple tests. It will later allow more flexible test configuration.\n",
    "\n",
    "See `tests/eas/acceptance.py` for an example of how this is currently used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Support to write files from Executor configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://github.com/ARM-software/lisa/pull/209\n",
    "\n",
    "A new \"files\" attribute can be added to Executor configurations which allows\n",
    "to specify a list files (e.g. sysfs and procfs) and values to be written to that files.\n",
    "\n",
    "For example, the following test configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tests_conf = {\n",
    "    \"confs\" : [\n",
    "        {\n",
    "            \"tag\"            : \"base\",\n",
    "            \"flags\"          : \"ftrace\",\n",
    "            \"sched_features\" : \"NO_ENERGY_AWARE\",\n",
    "            \"cpufreq\"        : {\n",
    "                \"governor\" : \"performance\",\n",
    "            },\n",
    "            \"files\"          : {\n",
    "                '/proc/sys/kernel/sched_is_big_little'           : '0',\n",
    "                '!/proc/sys/kernel/sched_migration_cost_ns'      : '500000'\n",
    "            },\n",
    "        }\n",
    "    ]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "can be used to run a test where the platform is configured to\n",
    "- disable the \"sched_is_big_little\" flag (if present)\n",
    "- set to 50ms the \"sched_migration_cost_ns\"\n",
    "\n",
    "Nortice that a value written in a file is verified only if the file path is\n",
    "**prefixed** by a '/'. Otherwise, the write never fails, e.g. if the file does not exists."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Support to freeze user-space across a test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://github.com/ARM-software/lisa/pull/227\n",
    "\n",
    "Executor learned the `\"freeze_userspace\"` conf flag. When this flag is present, LISA uses the devlib freezer to freeze as much of userspace as possible while the experiment workload is executing, in order to reduce system noise.\n",
    "\n",
    "The `Executor` example notebook:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/ipynb/examples/utils/executor_example.ipynb\n",
    "\n",
    "gives an example of using this feature."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Trace module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tasks name pre-loading"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the Trace module is initialized, by default all the tasks in that trace are identified and exposed via the usual getTask() method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-12-15 17:04:14,988 INFO    : Trace        : Parsing FTrace format...\n",
      "2016-12-15 17:04:15,162 INFO    : Trace        : Collected events spans a 11.783 [s] time interval\n",
      "2016-12-15 17:04:15,162 INFO    : Trace        : Set plots time range to (0.000000, 11.782618)[s]\n",
      "2016-12-15 17:04:15,163 INFO    : Analysis     : Registering trace analysis modules:\n",
      "2016-12-15 17:04:15,163 INFO    : Analysis     :    latency\n",
      "2016-12-15 17:04:15,193 INFO    : Analysis     :    eas\n",
      "2016-12-15 17:04:15,197 INFO    : Analysis     :    tasks\n",
      "2016-12-15 17:04:15,205 INFO    : Analysis     :    cpus\n",
      "2016-12-15 17:04:15,209 INFO    : Analysis     :    functions\n",
      "2016-12-15 17:04:15,212 INFO    : Analysis     :    status\n",
      "2016-12-15 17:04:15,214 INFO    : Analysis     :    idle\n",
      "2016-12-15 17:04:15,218 INFO    : Analysis     :    stune\n",
      "2016-12-15 17:04:15,224 INFO    : Analysis     :    frequency\n"
     ]
    }
   ],
   "source": [
    "from trace import Trace\n",
    "import json\n",
    "\n",
    "with open('/home/patbel01/Code/lisa/results/LisaInANutshell_Backup/platform.json', 'r') as fh:\n",
    "    platform = json.load(fh)\n",
    "\n",
    "trace = Trace(platform,\n",
    "    '/home/patbel01/Code/lisa/results/LisaInANutshell_Backup/trace.dat',\n",
    "    events=['sched_switch']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-12-15 17:05:03,760 INFO    : root         : 45 tasks loaded from trace\n"
     ]
    }
   ],
   "source": [
    "logging.info(\"%d tasks loaded from trace\", len(trace.getTasks()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-12-15 17:05:49,699 INFO    : root         : The rt-app task in this trace has this PID:\n",
      "2016-12-15 17:05:49,699 INFO    : root         :   {'pid': [1388, 1387]}\n"
     ]
    }
   ],
   "source": [
    "logging.info(\"The rt-app task in this trace has these PIDs:\")\n",
    "logging.info(\"  %s\", trace.getTasks()['rt-app'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Android Support"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Added support for Pixel Phones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new platform definition file has been added which allows to easily setup\n",
    "a connection with an Pixel device:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "    \"board\" : {\r\n",
      "        \"cores\" : [\r\n",
      "            \"Silver\", \"Silver\", \"Gold\", \"Gold\"\r\n",
      "        ],\r\n",
      "        \"big_core\" : \"Gold\",\r\n",
      "        \"modules\" : [\"bl\", \"cpufreq\"]\r\n",
      "    },\r\n",
      "    \"nrg_model\" : {\r\n",
      "        \"little\" : {\r\n",
      "            \"cpu\" : {\r\n",
      "                \"nrg_max\" :  925,\r\n",
      "                \"cap_max\" :  763,\r\n",
      "            },\r\n",
      "            \"cluster\" : {\r\n",
      "                \"nrg_max\" :  52\r\n",
      "            }\r\n",
      "        },\r\n",
      "        \"big\": {\r\n",
      "            \"cpu\" : {\r\n",
      "                \"nrg_max\" :  1715,\r\n",
      "                \"cap_max\" :  1024\r\n",
      "            },\r\n",
      "            \"cluster\" : {\r\n",
      "                \"nrg_max\" :  96\r\n",
      "            }\r\n",
      "        }\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!cat $LISA_HOME/libs/utils/platforms/pixel.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2016-12-15 17:15:15,670 INFO    : TestEnv      : Using base path: /home/patbel01/Code/lisa\n",
      "2016-12-15 17:15:15,671 INFO    : TestEnv      : Loading custom (inline) target configuration\n",
      "2016-12-15 17:15:15,671 INFO    : TestEnv      : External tools using:\n",
      "2016-12-15 17:15:15,671 INFO    : TestEnv      :    ANDROID_HOME: /home/patbel01/Code/lisa/tools/android-sdk-linux/\n",
      "2016-12-15 17:15:15,672 INFO    : TestEnv      :    CATAPULT_HOME: /home/patbel01/Code/lisa/tools/catapult\n",
      "2016-12-15 17:15:15,672 INFO    : TestEnv      : Loading board:\n",
      "2016-12-15 17:15:15,672 INFO    : TestEnv      :    /home/patbel01/Code/lisa/libs/utils/platforms/pixel.json\n",
      "2016-12-15 17:15:15,673 INFO    : TestEnv      : Devlib modules to load: [u'bl', u'cpufreq']\n",
      "2016-12-15 17:15:15,674 INFO    : TestEnv      : Connecting Android target [DEFAULT]\n",
      "2016-12-15 17:15:15,674 INFO    : TestEnv      : Connection settings:\n",
      "2016-12-15 17:15:15,674 INFO    : TestEnv      :    None\n",
      "2016-12-15 17:15:15,733 INFO    : android      : ls command is set to ls\n",
      "2016-12-15 17:15:16,068 INFO    : TestEnv      : Initializing target workdir:\n",
      "2016-12-15 17:15:16,068 INFO    : TestEnv      :    /data/local/tmp/devlib-target\n",
      "2016-12-15 17:15:16,555 INFO    : TestEnv      : Topology:\n",
      "2016-12-15 17:15:16,556 INFO    : TestEnv      :    [[0, 1], [2, 3]]\n",
      "2016-12-15 17:15:16,635 INFO    : TestEnv      : Loading default EM:\n",
      "2016-12-15 17:15:16,636 INFO    : TestEnv      :    /home/patbel01/Code/lisa/libs/utils/platforms/pixel.json\n",
      "2016-12-15 17:15:16,637 INFO    : TestEnv      : Set results folder to:\n",
      "2016-12-15 17:15:16,637 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results/20161215_171516\n",
      "2016-12-15 17:15:16,638 INFO    : TestEnv      : Experiment results available also in:\n",
      "2016-12-15 17:15:16,638 INFO    : TestEnv      :    /home/patbel01/Code/lisa/results_latest\n"
     ]
    }
   ],
   "source": [
    "from env import TestEnv\n",
    "\n",
    "te = TestEnv({\n",
    "        'platform'     : 'android',\n",
    "        'board'        : 'pixel',\n",
    "        'ANDROID_HOME' : '/home/patbel01/Code/lisa/tools/android-sdk-linux/'\n",
    "    }, force_new=True)\n",
    "target = te.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Added UiBench workload"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new Android benchmark has been added to run UiBench provided tests.\n",
    "Here is a notebook which provides an example of how to run this test on your\n",
    "android target:\n",
    "\n",
    "   https://github.com/ARM-software/lisa/blob/master/ipynb/examples/android/benchmarks/Android_UiBench.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intial version of the preliminary tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preliminary tests aim at verifying some basic support required for a\n",
    "complete functional EAS solution.\n",
    "\n",
    "A initial version of these preliminary tests is now available:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/tests/eas/preliminary.py\n",
    "\n",
    "and it will be extended in the future to include more and more tests."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Capacity capping test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new test has been added to verify that capacity capping is working\n",
    "as expected:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/tests/eas/capacity_capping.py\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Acceptance tests reworked"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The EAS acceptace test collects a set of platform independent tests to verify\n",
    "basic EAS beahviours.\n",
    "\n",
    "This test has been cleaned up and it's now avaiable with a detailed documentation:\n",
    "\n",
    "https://github.com/ARM-software/lisa/blob/master/tests/eas/acceptance.py\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebooks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Added scratchpad notebooks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new **scratchpad** folder has been added under the ipynb folder which collects the available notebooks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;34m/home/patbel01/Code/lisa/ipynb\u001b[00m\r\n",
      "├── \u001b[01;34mchromeos\u001b[00m\r\n",
      "├── \u001b[01;34mdevlib\u001b[00m\r\n",
      "├── \u001b[01;34menergy\u001b[00m\r\n",
      "├── \u001b[01;34mexamples\u001b[00m\r\n",
      "├── \u001b[01;34mprofiling\u001b[00m\r\n",
      "├── \u001b[01;34mreleases\u001b[00m\r\n",
      "├── \u001b[01;34msched_dvfs\u001b[00m\r\n",
      "├── \u001b[01;34msched_tune\u001b[00m\r\n",
      "├── \u001b[01;34mscratchpad\u001b[00m\r\n",
      "├── server.log\r\n",
      "├── server.pid\r\n",
      "├── \u001b[01;34mthermal\u001b[00m\r\n",
      "├── \u001b[01;34mtutorial\u001b[00m\r\n",
      "└── \u001b[01;34mwlgen\u001b[00m\r\n",
      "\r\n",
      "12 directories, 2 files\r\n"
     ]
    }
   ],
   "source": [
    "!tree -L 1 ~/Code/lisa/ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This folder is configured to be ignored by git, thus it's the best place to place your work-in-progress notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example notebook restructoring"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example notebooks has been consolidated and better organized by topic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;34m/home/patbel01/Code/lisa/ipynb/examples\u001b[00m\r\n",
      "├── \u001b[01;34mandroid\u001b[00m\r\n",
      "├── \u001b[01;34mdevlib\u001b[00m\r\n",
      "├── \u001b[01;34menergy_meter\u001b[00m\r\n",
      "├── \u001b[01;34mtrace_analysis\u001b[00m\r\n",
      "├── \u001b[01;34mtrappy\u001b[00m\r\n",
      "├── \u001b[01;34mutils\u001b[00m\r\n",
      "└── \u001b[01;34mwlgen\u001b[00m\r\n",
      "\r\n",
      "7 directories, 0 files\r\n"
     ]
    }
   ],
   "source": [
    "!tree -L 1 ~/Code/lisa/ipynb/examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the folder to look into when it comes to undedrstand how a specific\n",
    "LISA API works.\n",
    "\n",
    "Here is where we will provide a dedicated folder and set of notebooks for each of the main LISA modules."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  },
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}