Add arm64 build workflow to CI

[lloeki]

Motivation: catch regressions such as #1349 early

Workflow will intentionally not run tests as it'd probably be ungodly slow due to aarch64-on-x86_64 emulation via QEMU.

[smola]

Workflows to run a light scenario in macOS and Windows would also be nice. There's some bits that tend to break there too (although, fortunately, these do not change frequently).