commit | d1add6c90cbb3276b6d2fdcaffd00f03d5a95d14 | [log] [tgz] |
---|---|---|
author | Joel Fernandes <joelaf@google.com> | Tue Oct 31 21:34:36 2017 -0700 |
committer | Joel Fernandes <joelaf@google.com> | Tue Oct 31 23:26:58 2017 -0700 |
tree | 0930c48356065a96b561800f9c884fdb7f367bd1 | |
parent | 6b9d47532f12b7e5a9fd31104ced5ef9d7c69987 [diff] |
tools: Add a tool to compare Janbench results Its often difficult to find regressions in jankbench results visually. This tool allows to compare multiple test results between 2 different kernels or test runs. In the output of this tool, each statistic gets a _diff column which shows the delta between baseline and the test. Negative values in the diff column mean improvements (lower frame durations). Example run: ./compare_jankbench.py --baseline='./results/Jankbench_baseline' --compare-with='./results/Jankbench_kernel_change' The output will be something like (only showing 25% and 50%): 25% compare 25%_diff 50%_compare 50%_diff test_name image_list_view 2.11249 0.0178108 5.7952 0.0242445 list_view 2.02227 -3.65839 5.74957 -0.095421 shadow_grid 6.00877 -0.000898 6.23746 -0.0057695 high_hitrate_text 5.81625 0.0264913 6.03504 0.0017795 Change-Id: I2ba9e43309ab93b11059563e8db014ecc28c3fc7 Signed-off-by: Joel Fernandes <joelaf@google.com>
NOTE: This is still a work in progress project, suitable for: developers, contributors and testers. None of the provided tests have been extensively evaluated as of January 2017.
The LISA project provides a toolkit that supports regression testing and interactive analysis of Linux kernel behavior. LISA stands for Linux Integrated/Interactive System Analysis. LISA's goal is to help Linux kernel developers to measure the impact of modifications in core parts of the kernel. The focus is on the scheduler (e.g. EAS), power management and thermal frameworks. However LISA is generic and can be used for other purposes too.
LISA has a “host”/“target” model. LISA itself runs on a host machine, and uses the devlib toolkit to interact with the target via SSH, ADB or telnet. LISA is flexible with regard to the target OS; its only expectation is a Linux kernel-based system. Android, GNU/Linux and busybox style systems have all been used.
LISA provides features to describe workloads (notably using rt-app) and run them on targets. It can collect trace files from the target OS (e.g. systrace and ftrace traces), parse them via the TRAPpy framework. These traces can then be parsed and analysed in order to examine detailed target behaviour during the workload's execution.
Some LISA features may require modifying the target OS. For example, in order to collect ftrace files the target kernel must have CONFIG_DYNAMIC_FTRACE enabled.
There are two “entry points” for running LISA:
Via the Jupyter/IPython notebook framework. This allows LISA to be used interactively and supports visualisation of trace data. Some notebooks are provided with example and ready-made LISA use-cases.
Via the automated test framework. This framework allows the development of automated pass/fail regression tests for kernel behaviour. The BART toolkit provides additional domain-specific test assertions for this use-case. LISA provides some ready-made automated tests under the tests/
directory.
The main goals of LISA are:
More formal API documentation for LISA is a work in progress, however much of the API is currently described in the provided tutorial Jupyter notebooks.
This project is licensed under Apache-2.0.
This project includes some third-party code under other open source licenses. For more information, see lisa/tools/LICENSE.*
Contributions are accepted under Apache-2.0. Only submit contributions where you have authored all of the code. If you do this on work time make sure your employer is cool with this.