--verify fails for many addresses in a solib sometimes.

[nate-thirdwave]

We have an internal project that compiles a C++ shared library and links it into several test binaries. Noteably though, the solib is a single compilation with -g and -O0. Link-time optimization is not used.

Running each of these binaries under kcov, sometimes the address registrations for the solib are fully accepted. The other test binary run finds that many address mappings fail with text similar to:

kcov: Address 0x7497 is not at an instruction boundary, skipping

This seems to be emitted as a direct result of failing the verification step in elf-parser.cc:427

The compiler being used is gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 (ubuntu LTS), so while it's possible this is the manifestation of a compiler bug, I am inclined to suspect the bfd-disassembler.

What I am at a loss to understand though is how the solib succeeds verification in one context, but not the other. Also noteable, even the failure does not mean all lines fail verification. Some lines are marked as valid and receive coverage data, though also some lines are unexpectedly marked as instrumented. This results in confusing and incorrect kcov coverage reports.

The incorrectly instrumented lines appear to be stable from program run to run.

kcov version adc5653 (HEAD as of this issue writing)

The current workaround appears to be not using --verify, and the expected instrumented and coverage results appear.

[SimonKagstrom]

OK, this is an interesting variant of the "usual" problem of having to use --verify to workaround broken DWARF information.

I agree it sounds like an issue with the BFD disassembler, unless bfd-disassembler.cpp does something wrong. Do you have more information about the instruction context which causes it to skip?

Variable-length instruction sets are unfortunate, but I'll guess we'll have to live with them as long as x86 is around (i.e., probably until well after I'm retired :-)).