GPT table in Genimage-created images may be mangled by Windows OS

[sairon]

Windows (at least starting with version 10) contain logic that alters the GPT table of a drive connected to it in some cases. More specifically it's been found it relocates the backup GPT table to the end of the drive and the partition array to LBA address equal to first_usable_lba - 32. There is no (easy) way to prevent this behavior in Genimage-created images because first_usable_lba is always set to the start of the first partition. Although this is rather a Windows bug/misfeature, images created using other utilities (like sgdisk) are not affected by this, as they generally set first_usable_lba to LBA 34, so it should be possible to achieve the same with Genimage.

For details see:

• HAOS 12.4 install created on Windows 10+ fails to boot on RPi 5 home-assistant/operating-system#3437
• RPi 5 bootloader doesn't load kernel with GPT partition array not at LBA 2 raspberrypi/rpi-eeprom#585

These issues show a real world regression triggered by the fact that Genimage works slightly different compared to other utilities. In theory I can imagine this relocation might also overwrite data located between the GPT header and the first actual partition if this area of the disk contained for example some bootloader data - although this is purely hypothetical.

I'd like to discuss the potential ways to mitigate this issue. Should we add a flag for setting first_usable_lba to the lowest possible value, or an arbitrary one, or even set it always to 34 like sgdisk does? Currently it's correlated to the align value but setting it to a one-sector size and juggling with offset values afterwards feels rather clumsy to me.

[michaelolbrich]

So my understanding was that first_usable_lba is the first LBA that can be used for partitions, and anything before that is off limits. That's why it is implemented as it is right now: There are real use-cases where bootloader data is placed between the partition table and the first partition.

But my understanding is clearly mistaken, or at least windows has a different understanding of what should happen here. So we should probably always do:

header.first_usable_lba = hd->gpt_location + (GPT_SECTORS - 1)*512;

That's 2 + 32 = 34 for the default partition table location. If the partition table is moved to make room for a bootloader, then that should still be acceptable based on you're description above.

[sairon]

Hi Michael, first of all, sorry for the delayed reply.

So my understanding was that first_usable_lba is the first LBA that can be used for partitions, and anything before that is off limits. That's why it is implemented as it is right now: There are real use-cases where bootloader data is placed between the partition table and the first partition.

Well, I'd assume the same but obviously Microsoft engineers thought it's good idea to do it like this. I haven't found any reasoning why it's happening, and I think it's too esoteric to cause any other issues. Just find the cause of the problem was quite a detective work.

But my understanding is clearly mistaken, or at least windows has a different understanding of what should happen here. So we should probably always do:
...

It's essentially the same what I ended up with: home-assistant/operating-system#3497 (just taking into the fact that header.first_usable_lba is actually the LBA number, not byte offset). So if you agree, I'll put it into a PR here.

FWIW, the root cause for the RPi bootloader issue has been found and fix is on the way, but I think it'd be still good to stay on the safe side and adjust the Genimage behavior, as I'm not aware of any side effects.