Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor the instruction tracing subsystem. #168

Open
wants to merge 74 commits into
base: dev
Choose a base branch
from
Open

Refactor the instruction tracing subsystem. #168

wants to merge 74 commits into from
+4,969 −796

Conversation

qwattash
Copy link
Contributor

Add tracing backend based on the google perfetto tracing framework.
Add the ability to aggregate statistics within qemu and emit them as part of the perfetto backend.
Add support for generic events associated to instructions.
Add better support for trace filters for each log entry.

qwattash
qwattash commented Sep 29, 2021
View reviewed changes
configure Outdated Show resolved Hide resolved
arichardson
arichardson reviewed Sep 29, 2021
View reviewed changes
Copy link
Member

@arichardson arichardson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This generally looks fine to me (only skimmed most of the commits though). Since it's off by default it shouldn't really matter that we pull in boost.

accel/tcg/log_instr.c Outdated Show resolved Hide resolved
include/exec/log_instr.h Outdated Show resolved Hide resolved
target/riscv/op_helper_log_instr.c Outdated Show resolved Hide resolved
stats/meson.build Outdated Show resolved Hide resolved
@qwattash qwattash changed the base branch from qemu-cheri to dev November 20, 2021 00:56
@qwattash qwattash force-pushed the qemu-experimental-tracing branch 2 times, most recently from b0cd6a4 to 9b1e0ee Compare April 26, 2022 19:43
@qwattash qwattash force-pushed the qemu-experimental-tracing branch 2 times, most recently from 7bd5a53 to f7417a1 Compare August 18, 2022 14:21
@qwattash
Refactor instruction tracing to support events.
d1a2044 
- Rename cpu_log_instr_info to cpu_log_entry.
@qwattash
Refactor instruction logging: rename trace_format to trace_backend.
aca9967
@qwattash
Implement an event queue for each instruction entry logged.
de64762
@qwattash
Generalize instruction trace filtering.
89ece23 
The -dfilter qemu option has ad-hoc support within TCG instruction tracing,
this generalizes the trace filtering to an arbitrary per-CPU set of functions
that can filter the trace entry based on any parameter.
Note that for now filters are assumed to be stateless.
@qwattash
Refactor TCG log instruction backend into multiple files.
1645382 
Each trace format is moved to a separate file for implementation.
@qwattash
Actually emit events.
a5ab240
@qwattash
Rename TCG instruction logging backend macros.
05fb03a
@qwattash
Add magic NOP to log context switch events.
ce35201
@qwattash
Add stats TCG instruction logging backend.
d8b6486 
This backend is intended to capture statistics from the instructions
instead of dumping the whole instruction stream.
Note that this is disabled by default in the configuration as it adds
a dependency on libboost.
@qwattash
Add initialization and setup code for the perfetto tracing backend.
edc0a74 
- Add cheri-perfetto submodule for tracing.
- Plumb the perfetto tracing backend in the build system
- Add initialization code that registers the tracing categories and starts
  the tracing service at startup
- Add skeleton for the perfetto tracing backend glue code
@qwattash
Fix stats logging backend output.
c88434e
@qwattash
Improve instruction trace filters support.
b809581 
- Add command line option to select filters to use.
- Make sure that the filters from command line are applied to the CPUs at
  initialization.
- Add filter to only emit entries containing events.
@qwattash
Refactor trace stats helper into a class.
2ed217d
@qwattash
Fix broken legacy CVTrace backend.
cd44395
@qwattash
Implement a tracing backend based on the perfetto tracing framework.
a31419e 
For each CPU we create 2 tracks: one for control events (e.g. tracing
start/stop) and another for log events on that CPU.
Currently we emit instructions and stats events on the CPU track.

- Replace the stats backend with a tracing category in the perfetto backend
- Add new CLI options to control the perfetto backend behaviour:
  1. --cheri-trace-perfetto-categories comma-separated list of categories to
     enable.
  2. --cheri-trace-perfetto-logfile log file to use for the perfetto binary
     trace. We do not use the common qemu logfile to avoid accidental pollution
     and passing the qemu file descriptor is painful due to having to lock the
     logfile via RCU.
- Add hook at qemu shutdown to make sure we flush the log buffers, if the
  tracing backend supports this.
@qwattash
Fix flushing in instruction logging perfetto backend.
5b961f1 
Flush TrackEvent data source when we receive a FLUSH event in the instruction
logging subsystem. This helps making sure that we do not have lost events when
running multiple tests on qemu.
qwattash and others added 20 commits January 30, 2023 22:20
@qwattash
Implement register dumps for protobuf instruction tracing backend.
47348d2
@qwattash
Retire instruction logging event type for context updates.
49abdf2 
Remove the event variant that bypasses log enable. This is useless as
switching on/off tracing around the event is feasible and the event is
rare enough that it is not a performance issue.
@qwattash
Implement RISC-V tracing counter NOPs.
7dcf684 
Add NOP that guests can use to create and increment/decrement
counters.
The counters are emitted into the perfetto trace stream as normal
perfetto counter tracks.
@qwattash
Add per-context counters.
692ea29 
These are still not hooked to the guest NOPs.
@qwattash
Point cheri-perfetto submodule to the correct repository.
ec20ea4
@qwattash
Introduce interval tracks support in the perfetto tracing backend.
1cdf7bc 
Replace the stats implementation with a new one based on the
cheri-perfetto interval tracks.
The stats implementation tracks basic block icount and hits, along
with branches. Now there are multiple categories for each different
statistic instead of a grouped one.
@qwattash
Fix instruction opcode dumping to the perfetto message format.
719e787 
This works around the inability to use bytes fields in perfetto
track event messages.
@qwattash
Fix cheri-perfetto new remote URL.
2d86daf
@qwattash
Add option to select output file for the protobuf tracing backend.
5e5a6da
@qwattash
Unbreak -icount mode for RISC-V.
674fc6e
@qwattash
Add an experimental interceptor in perfetto to trace memory load and …
7576074 
…store
@qwattash
add gzip compression for interceptor output file
7fb7bfa
@qwattash
Point dynamrio submodule to CHERI fork
746fca3
@qwattash
Drop the instruction tracing backend emit_header hook.
1ea363c 
This was essentially unused, and can be replicated during initialization.
@qwattash
Improve perfetto trace session configuration.
8c80e97
@qwattash
Introduce support for logging qemu debugging counters with perfetto.
385939d 
Add two counters to measure instruction count when tracing is enabled/disabled.
@qwattash
Disable branch and basic block tracking when unnecessary.
6a44e1d 
Only maintain the branch and basic block hit histograms when
the corresponding trace categories are enabled.
@qwattash
Disable tb_flush when switching log level.
9660d3b 
This should not be needed because we have flags marking TBs whenever
they contain instruction tracing instrumentation.
@qwattash
Fix io_uring_prep_poll_remove call.
717464c 
This is broken by recent liburing versions.
@qwattash
Always use a fixed number of entries in the trace backends list.
82b638f
@qwattash
Update disasm helpers to emit single instructions as strings.
6a9bbaa 
Add disas_one_strbuf() to disassemble one instruction to an existing
string buffer.
Add disas_one_str() to disassemble one instruction to a dynamically
allocated string.
@qwattash
Use disas_one_str in protobuf backend.
e952789
@qwattash
Add explicit file parameter to non-text tracing backends.
b1ba45d 
- Conditionally compile cvtrace
- This is done because some of these backends emit binary data that
  is logically incompatible with the standard qemu output file.
  The downside is the we need to handle our own locking if we
  want to support multiple CPUs.
@qwattash
Cleanup signatures in instruction logging infrastructure.
a2ac590
@qwattash
Add the current log level transition in the log state transition event.
659bcc5 
This should help to inform log backends for management purposes.
@qwattash
Rename per-CPU instruction logging buffer to entry_buffer.
9ae03c6 
This is better than instr_info as it reflects its use.
@qwattash
Remove perfetto debug counters.
1d91718
@qwattash
Restore entry buffer size var.
d83469d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arichardson arichardson arichardson left review comments

@kwitaszczyk kwitaszczyk Awaiting requested review from kwitaszczyk

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@qwattash @arichardson