Perfetto key concepts and architecture
Producer <> Service <> Consumer model
The tracing service is a long-lived entity (a system daemon on Linux/Android, a service in Chrome) that has the following responsibilities:
- Maintains a registry of active producers and their data sources.
- Owns the trace buffers.
- Handles multiplexing of several tracing sessions.
- Routes the trace config from the consumers to the corresponding producers.
- Tells the Producers when and what to trace.
- Moves data from the Producer's shared memory buffer to the central non-shared trace buffers.
A producer is an untrusted entity that offers the ability to contribute to the trace. In a multiprocess model, a producer almost always corresponds to a client process of the tracing service. It advertises its ability to contribute to the trace with one or more data sources. Each producer has exactly:
- One shared memory buffer, shared exclusively with the tracing service.
- One IPC channel with the tracing service.
A producer is completely decoupled (both technically and conceptually) from consumer(s). A producer knows nothing about:
- How many consumer(s) are connected to the service.
- How many tracing sessions are active.
- How many other producer(s) are registered or active.
- Trace data written by other producer(s).
In rare circumstances a process can host more than one producer and hence more than one shared memory buffer. This can be the case for a process bundling third-party libraries that in turn include the Perfetto client library.
Concrete example: at some point in the future Chrome might expose one Producer for tracing within the main project, one for V8 and one for Skia (for each child process).
A consumer is a trusted entity (a cmdline client on Linux/Android, an interface of the Browser process in Chrome) that controls (non-exclusively) the tracing service and reads back (destructively) the trace buffers. A consumer has the ability to:
- Send a trace config to the service, determining:
- How many trace buffers to create.
- How big the trace buffers should be.
- The policy for each buffer (ring-buffer or stop-when-full).
- Which data sources to enable.
- The configuration for each data source.
- The target buffer for the data produced by each data source configured.
- Enable and disable tracing.
- Read back the trace buffers:
- Streaming data over the IPC channel.
- Passing a file descriptor to the service and instructing it to periodically save the trace buffers into the file.
A data source is a capability, exposed by a Producer, of providing some tracing data. A data source almost always defines its own schema (a protobuf) consisting of:
- At most one
DataSourceConfig sub-message (example)
- One or more
TracePacket sub-messages (example)
Different producers may expose the same data source. Concrete example:
At some point in the near future we might offer, as part of Perfetto, a library for in-process heap profiling. In such case more than one producer, linking against the updated Perfetto library, will expose the heap profiler data source, for its own process.
In a multiprocess scenario, each producer and each consumer interact with the service using an IPC channel. IPC is used only in non-fast-path interactions, mostly handshakes such as enabling/disabling trace (consumer), (un)registering and starting/stopping data sources (producer). The IPC is typically NOT employed to transport the protobufs for the trace. Perfetto provides a POSIX-friendly IPC implementation, based on protobufs over a UNIX socket (see ipc.md). That IPC implementation is not mandated. Perfetto allows the embedder:
- Wrap its own IPC subsystem (e.g., Perfetto in Chromium uses Mojo)
- Not use an IPC mechanism at all and just short circuit the Producer <> Service <> Consumer interaction via
See embedder-guide.md for more details.
Shared memory buffer
Producer(s) write tracing data, in the form of protobuf-encoded binary blobs, directly into its shared memory buffer, using a special library called ProtoZero. The shared memory buffer:
- Has a fixed and typically small size (configurable, default: 128 KB).
- Is an ABI and must maintain backwards compatibility.
- Is shared by all data sources of the producer.
- Is independent of the number and the size of the trace buffers.
- Is independent of the number of Consumer(s).
- Is partitioned in chunks of variable size.
- Is owned exclusively by one Producer thread (or shared through a mutex).
- Contains a linear sequence of
TracePacket(s), or fragments of that. A
TracePacket can span across several chunks, the fragmentation is not exposed to the consumers (consumers always see whole packets as if they were never fragmented).
- Can be owned and written by exactly one
- Is part of a reliable and ordered sequence, identified by the
WriterID: packets in a sequence are guaranteed to be read back in order, without gaps and without repetitions (see trace-format.md for more).
See the comments in shared_memory_abi.h for more details about the binary format of this buffer.