| |
| <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
| <html><head><title>Python: module telemetry.benchmark</title> |
| <meta charset="utf-8"> |
| </head><body bgcolor="#f0f0f8"> |
| |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="heading"> |
| <tr bgcolor="#7799ee"> |
| <td valign=bottom> <br> |
| <font color="#ffffff" face="helvetica, arial"> <br><big><big><strong><a href="telemetry.html"><font color="#ffffff">telemetry</font></a>.benchmark</strong></big></big></font></td |
| ><td align=right valign=bottom |
| ><font color="#ffffff" face="helvetica, arial"><a href=".">index</a><br><a href="../telemetry/benchmark.py">telemetry/benchmark.py</a></font></td></tr></table> |
| <p><tt># Copyright 2014 The Chromium Authors. All rights reserved.<br> |
| # Use of this source code is governed by a BSD-style license that can be<br> |
| # found in the LICENSE file.</tt></p> |
| <p> |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section"> |
| <tr bgcolor="#aa55cc"> |
| <td colspan=3 valign=bottom> <br> |
| <font color="#ffffff" face="helvetica, arial"><big><strong>Modules</strong></big></font></td></tr> |
| |
| <tr><td bgcolor="#aa55cc"><tt> </tt></td><td> </td> |
| <td width="100%"><table width="100%" summary="list"><tr><td width="25%" valign=top><a href="telemetry.internal.util.command_line.html">telemetry.internal.util.command_line</a><br> |
| <a href="telemetry.decorators.html">telemetry.decorators</a><br> |
| </td><td width="25%" valign=top><a href="optparse.html">optparse</a><br> |
| <a href="telemetry.page.page_test.html">telemetry.page.page_test</a><br> |
| </td><td width="25%" valign=top><a href="telemetry.internal.story_runner.html">telemetry.internal.story_runner</a><br> |
| <a href="telemetry.web_perf.timeline_based_measurement.html">telemetry.web_perf.timeline_based_measurement</a><br> |
| </td><td width="25%" valign=top></td></tr></table></td></tr></table><p> |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section"> |
| <tr bgcolor="#ee77aa"> |
| <td colspan=3 valign=bottom> <br> |
| <font color="#ffffff" face="helvetica, arial"><big><strong>Classes</strong></big></font></td></tr> |
| |
| <tr><td bgcolor="#ee77aa"><tt> </tt></td><td> </td> |
| <td width="100%"><dl> |
| <dt><font face="helvetica, arial"><a href="__builtin__.html#object">__builtin__.object</a> |
| </font></dt><dd> |
| <dl> |
| <dt><font face="helvetica, arial"><a href="telemetry.benchmark.html#BenchmarkMetadata">BenchmarkMetadata</a> |
| </font></dt></dl> |
| </dd> |
| <dt><font face="helvetica, arial"><a href="exceptions.html#Exception">exceptions.Exception</a>(<a href="exceptions.html#BaseException">exceptions.BaseException</a>) |
| </font></dt><dd> |
| <dl> |
| <dt><font face="helvetica, arial"><a href="telemetry.benchmark.html#InvalidOptionsError">InvalidOptionsError</a> |
| </font></dt></dl> |
| </dd> |
| <dt><font face="helvetica, arial"><a href="telemetry.internal.util.command_line.html#Command">telemetry.internal.util.command_line.Command</a>(<a href="telemetry.internal.util.command_line.html#ArgumentHandlerMixIn">telemetry.internal.util.command_line.ArgumentHandlerMixIn</a>) |
| </font></dt><dd> |
| <dl> |
| <dt><font face="helvetica, arial"><a href="telemetry.benchmark.html#Benchmark">Benchmark</a> |
| </font></dt></dl> |
| </dd> |
| </dl> |
| <p> |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section"> |
| <tr bgcolor="#ffc8d8"> |
| <td colspan=3 valign=bottom> <br> |
| <font color="#000000" face="helvetica, arial"><a name="Benchmark">class <strong>Benchmark</strong></a>(<a href="telemetry.internal.util.command_line.html#Command">telemetry.internal.util.command_line.Command</a>)</font></td></tr> |
| |
| <tr bgcolor="#ffc8d8"><td rowspan=2><tt> </tt></td> |
| <td colspan=2><tt>Base class for a Telemetry benchmark.<br> |
| <br> |
| A benchmark packages a measurement and a PageSet together.<br> |
| Benchmarks default to using TBM unless you override the value of<br> |
| <a href="#Benchmark">Benchmark</a>.test, or override the CreatePageTest method.<br> |
| <br> |
| New benchmarks should override CreateStorySet.<br> </tt></td></tr> |
| <tr><td> </td> |
| <td width="100%"><dl><dt>Method resolution order:</dt> |
| <dd><a href="telemetry.benchmark.html#Benchmark">Benchmark</a></dd> |
| <dd><a href="telemetry.internal.util.command_line.html#Command">telemetry.internal.util.command_line.Command</a></dd> |
| <dd><a href="telemetry.internal.util.command_line.html#ArgumentHandlerMixIn">telemetry.internal.util.command_line.ArgumentHandlerMixIn</a></dd> |
| <dd><a href="__builtin__.html#object">__builtin__.object</a></dd> |
| </dl> |
| <hr> |
| Methods defined here:<br> |
| <dl><dt><a name="Benchmark-CreatePageTest"><strong>CreatePageTest</strong></a>(self, options)</dt><dd><tt>Return the PageTest for this <a href="#Benchmark">Benchmark</a>.<br> |
| <br> |
| Override this method for PageTest tests.<br> |
| Override, override CreateTimelineBasedMeasurementOptions to configure<br> |
| TimelineBasedMeasurement tests. Do not override both methods.<br> |
| <br> |
| Args:<br> |
| options: a browser_options.BrowserFinderOptions instance<br> |
| Returns:<br> |
| |<a href="#Benchmark-test">test</a>()| if |test| is a PageTest class.<br> |
| Otherwise, a TimelineBasedMeasurement instance.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-CreateStorySet"><strong>CreateStorySet</strong></a>(self, options)</dt><dd><tt>Creates the instance of StorySet used to run the benchmark.<br> |
| <br> |
| Can be overridden by subclasses.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-CreateTimelineBasedMeasurementOptions"><strong>CreateTimelineBasedMeasurementOptions</strong></a>(self)</dt><dd><tt>Return the TimelineBasedMeasurementOptions for this <a href="#Benchmark">Benchmark</a>.<br> |
| <br> |
| Override this method to configure a TimelineBasedMeasurement benchmark.<br> |
| Otherwise, override CreatePageTest for PageTest tests. Do not override<br> |
| both methods.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-CustomizeBrowserOptions"><strong>CustomizeBrowserOptions</strong></a>(self, options)</dt><dd><tt>Add browser options that are required by this benchmark.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-GetMetadata"><strong>GetMetadata</strong></a>(self)</dt></dl> |
| |
| <dl><dt><a name="Benchmark-GetTraceRerunCommands"><strong>GetTraceRerunCommands</strong></a>(self)</dt></dl> |
| |
| <dl><dt><a name="Benchmark-Run"><strong>Run</strong></a>(self, finder_options)</dt><dd><tt>Do not override this method.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-SetupBenchmarkDebugTraceRerunOptions"><strong>SetupBenchmarkDebugTraceRerunOptions</strong></a>(self, tbm_options)</dt><dd><tt>Setup tracing categories associated with debug trace option.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-SetupBenchmarkDefaultTraceRerunOptions"><strong>SetupBenchmarkDefaultTraceRerunOptions</strong></a>(self, tbm_options)</dt><dd><tt>Setup tracing categories associated with default trace option.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-SetupTraceRerunOptions"><strong>SetupTraceRerunOptions</strong></a>(self, browser_options, tbm_options)</dt></dl> |
| |
| <dl><dt><a name="Benchmark-__init__"><strong>__init__</strong></a>(self, max_failures<font color="#909090">=None</font>)</dt><dd><tt>Creates a new <a href="#Benchmark">Benchmark</a>.<br> |
| <br> |
| Args:<br> |
| max_failures: The number of story run's failures before bailing<br> |
| from executing subsequent page runs. If None, we never bail.</tt></dd></dl> |
| |
| <hr> |
| Class methods defined here:<br> |
| <dl><dt><a name="Benchmark-AddCommandLineArgs"><strong>AddCommandLineArgs</strong></a>(cls, parser)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt></dl> |
| |
| <dl><dt><a name="Benchmark-HasTraceRerunDebugOption"><strong>HasTraceRerunDebugOption</strong></a>(cls)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt></dl> |
| |
| <dl><dt><a name="Benchmark-Name"><strong>Name</strong></a>(cls)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt></dl> |
| |
| <dl><dt><a name="Benchmark-ProcessCommandLineArgs"><strong>ProcessCommandLineArgs</strong></a>(cls, parser, args)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt></dl> |
| |
| <dl><dt><a name="Benchmark-SetArgumentDefaults"><strong>SetArgumentDefaults</strong></a>(cls, parser)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt></dl> |
| |
| <dl><dt><a name="Benchmark-ShouldDisable"><strong>ShouldDisable</strong></a>(cls, possible_browser)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt><dd><tt>Override this method to disable a benchmark under specific conditions.<br> |
| <br> |
| Supports logic too complex for simple Enabled and Disabled decorators.<br> |
| Decorators are still respected in cases where this function returns False.</tt></dd></dl> |
| |
| <dl><dt><a name="Benchmark-ValueCanBeAddedPredicate"><strong>ValueCanBeAddedPredicate</strong></a>(cls, value, is_first_result)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt><dd><tt>Returns whether |value| can be added to the test results.<br> |
| <br> |
| Override this method to customize the logic of adding values to test<br> |
| results.<br> |
| <br> |
| Args:<br> |
| value: a value.Value instance (except failure.FailureValue,<br> |
| skip.SkipValue or trace.TraceValue which will always be added).<br> |
| is_first_result: True if |value| is the first result for its<br> |
| corresponding story.<br> |
| <br> |
| Returns:<br> |
| True if |value| should be added to the test results.<br> |
| Otherwise, it returns False.</tt></dd></dl> |
| |
| <hr> |
| Data descriptors defined here:<br> |
| <dl><dt><strong>max_failures</strong></dt> |
| </dl> |
| <hr> |
| Data and other attributes defined here:<br> |
| <dl><dt><strong>options</strong> = {}</dl> |
| |
| <dl><dt><strong>test</strong> = <class 'telemetry.web_perf.timeline_based_measurement.TimelineBasedMeasurement'><dd><tt>Collects multiple metrics based on their interaction records.<br> |
| <br> |
| A timeline based measurement shifts the burden of what metrics to collect onto<br> |
| the story under test. Instead of the measurement<br> |
| having a fixed set of values it collects, the story being tested<br> |
| issues (via javascript) an Interaction record into the user timing API that<br> |
| describing what is happening at that time, as well as a standardized set<br> |
| of flags describing the semantics of the work being done. The<br> |
| TimelineBasedMeasurement <a href="__builtin__.html#object">object</a> collects a trace that includes both these<br> |
| interaction records, and a user-chosen amount of performance data using<br> |
| Telemetry's various timeline-producing APIs, tracing especially.<br> |
| <br> |
| It then passes the recorded timeline to different TimelineBasedMetrics based<br> |
| on those flags. As an example, this allows a single story run to produce<br> |
| load timing data, smoothness data, critical jank information and overall cpu<br> |
| usage information.<br> |
| <br> |
| For information on how to mark up a page to work with<br> |
| TimelineBasedMeasurement, refer to the<br> |
| perf.metrics.timeline_interaction_record module.<br> |
| <br> |
| Args:<br> |
| options: an instance of timeline_based_measurement.Options.<br> |
| results_wrapper: A class that has the __init__ method takes in<br> |
| the page_test_results <a href="__builtin__.html#object">object</a> and the interaction record label. This<br> |
| class follows the ResultsWrapperInterface. Note: this class is not<br> |
| supported long term and to be removed when crbug.com/453109 is resolved.</tt></dl> |
| |
| <hr> |
| Class methods inherited from <a href="telemetry.internal.util.command_line.html#Command">telemetry.internal.util.command_line.Command</a>:<br> |
| <dl><dt><a name="Benchmark-Description"><strong>Description</strong></a>(cls)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt></dl> |
| |
| <dl><dt><a name="Benchmark-main"><strong>main</strong></a>(cls, args<font color="#909090">=None</font>)<font color="#909090"><font face="helvetica, arial"> from <a href="__builtin__.html#type">__builtin__.type</a></font></font></dt><dd><tt>Main method to run this command as a standalone script.</tt></dd></dl> |
| |
| <hr> |
| Data descriptors inherited from <a href="telemetry.internal.util.command_line.html#ArgumentHandlerMixIn">telemetry.internal.util.command_line.ArgumentHandlerMixIn</a>:<br> |
| <dl><dt><strong>__dict__</strong></dt> |
| <dd><tt>dictionary for instance variables (if defined)</tt></dd> |
| </dl> |
| <dl><dt><strong>__weakref__</strong></dt> |
| <dd><tt>list of weak references to the object (if defined)</tt></dd> |
| </dl> |
| </td></tr></table> <p> |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section"> |
| <tr bgcolor="#ffc8d8"> |
| <td colspan=3 valign=bottom> <br> |
| <font color="#000000" face="helvetica, arial"><a name="BenchmarkMetadata">class <strong>BenchmarkMetadata</strong></a>(<a href="__builtin__.html#object">__builtin__.object</a>)</font></td></tr> |
| |
| <tr><td bgcolor="#ffc8d8"><tt> </tt></td><td> </td> |
| <td width="100%">Methods defined here:<br> |
| <dl><dt><a name="BenchmarkMetadata-AsDict"><strong>AsDict</strong></a>(self)</dt></dl> |
| |
| <dl><dt><a name="BenchmarkMetadata-__init__"><strong>__init__</strong></a>(self, name, description<font color="#909090">=''</font>, rerun_options<font color="#909090">=None</font>)</dt></dl> |
| |
| <hr> |
| Data descriptors defined here:<br> |
| <dl><dt><strong>__dict__</strong></dt> |
| <dd><tt>dictionary for instance variables (if defined)</tt></dd> |
| </dl> |
| <dl><dt><strong>__weakref__</strong></dt> |
| <dd><tt>list of weak references to the object (if defined)</tt></dd> |
| </dl> |
| <dl><dt><strong>description</strong></dt> |
| </dl> |
| <dl><dt><strong>name</strong></dt> |
| </dl> |
| <dl><dt><strong>rerun_options</strong></dt> |
| </dl> |
| </td></tr></table> <p> |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section"> |
| <tr bgcolor="#ffc8d8"> |
| <td colspan=3 valign=bottom> <br> |
| <font color="#000000" face="helvetica, arial"><a name="InvalidOptionsError">class <strong>InvalidOptionsError</strong></a>(<a href="exceptions.html#Exception">exceptions.Exception</a>)</font></td></tr> |
| |
| <tr bgcolor="#ffc8d8"><td rowspan=2><tt> </tt></td> |
| <td colspan=2><tt>Raised for invalid benchmark options.<br> </tt></td></tr> |
| <tr><td> </td> |
| <td width="100%"><dl><dt>Method resolution order:</dt> |
| <dd><a href="telemetry.benchmark.html#InvalidOptionsError">InvalidOptionsError</a></dd> |
| <dd><a href="exceptions.html#Exception">exceptions.Exception</a></dd> |
| <dd><a href="exceptions.html#BaseException">exceptions.BaseException</a></dd> |
| <dd><a href="__builtin__.html#object">__builtin__.object</a></dd> |
| </dl> |
| <hr> |
| Data descriptors defined here:<br> |
| <dl><dt><strong>__weakref__</strong></dt> |
| <dd><tt>list of weak references to the object (if defined)</tt></dd> |
| </dl> |
| <hr> |
| Methods inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br> |
| <dl><dt><a name="InvalidOptionsError-__init__"><strong>__init__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__init__">__init__</a>(...) initializes x; see help(type(x)) for signature</tt></dd></dl> |
| |
| <hr> |
| Data and other attributes inherited from <a href="exceptions.html#Exception">exceptions.Exception</a>:<br> |
| <dl><dt><strong>__new__</strong> = <built-in method __new__ of type object><dd><tt>T.<a href="#InvalidOptionsError-__new__">__new__</a>(S, ...) -> a new <a href="__builtin__.html#object">object</a> with type S, a subtype of T</tt></dl> |
| |
| <hr> |
| Methods inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br> |
| <dl><dt><a name="InvalidOptionsError-__delattr__"><strong>__delattr__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__delattr__">__delattr__</a>('name') <==> del x.name</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__getattribute__"><strong>__getattribute__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__getattribute__">__getattribute__</a>('name') <==> x.name</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__getitem__"><strong>__getitem__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__getitem__">__getitem__</a>(y) <==> x[y]</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__getslice__"><strong>__getslice__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__getslice__">__getslice__</a>(i, j) <==> x[i:j]<br> |
| <br> |
| Use of negative indices is not supported.</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__reduce__"><strong>__reduce__</strong></a>(...)</dt></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__repr__"><strong>__repr__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__repr__">__repr__</a>() <==> repr(x)</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__setattr__"><strong>__setattr__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__setattr__">__setattr__</a>('name', value) <==> x.name = value</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__setstate__"><strong>__setstate__</strong></a>(...)</dt></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__str__"><strong>__str__</strong></a>(...)</dt><dd><tt>x.<a href="#InvalidOptionsError-__str__">__str__</a>() <==> str(x)</tt></dd></dl> |
| |
| <dl><dt><a name="InvalidOptionsError-__unicode__"><strong>__unicode__</strong></a>(...)</dt></dl> |
| |
| <hr> |
| Data descriptors inherited from <a href="exceptions.html#BaseException">exceptions.BaseException</a>:<br> |
| <dl><dt><strong>__dict__</strong></dt> |
| </dl> |
| <dl><dt><strong>args</strong></dt> |
| </dl> |
| <dl><dt><strong>message</strong></dt> |
| </dl> |
| </td></tr></table></td></tr></table><p> |
| <table width="100%" cellspacing=0 cellpadding=2 border=0 summary="section"> |
| <tr bgcolor="#eeaa77"> |
| <td colspan=3 valign=bottom> <br> |
| <font color="#ffffff" face="helvetica, arial"><big><strong>Functions</strong></big></font></td></tr> |
| |
| <tr><td bgcolor="#eeaa77"><tt> </tt></td><td> </td> |
| <td width="100%"><dl><dt><a name="-AddCommandLineArgs"><strong>AddCommandLineArgs</strong></a>(parser)</dt></dl> |
| <dl><dt><a name="-ProcessCommandLineArgs"><strong>ProcessCommandLineArgs</strong></a>(parser, args)</dt></dl> |
| </td></tr></table> |
| </body></html> |
