Arm64 bindings

[konistehrad]

Depends on #1824. Expands bindings to include ARM64 for Linux and Mac, and updates Mac headers to 12.3 to allow for ARM64 usage.

Uses the following updated header package: headers.zip

[konistehrad]

Sorry, meant to open as draft. I made a pretty silly mistake in the MacOS headers that I'll have to correct tomorrow. Would love some eyes on the linux side in the meantime tho.

[konistehrad]

Took a crack at resolving the errors this morning to no avail. Updated the headers package with a clean version. Might help to have someone more familiar with the project take a swing at resolving the segfaults on ARM64 Mac.

[tritao]

This looks pretty close, think you might need to update here aswell: https://github.com/mono/CppSharp/blob/main/src/CppParser/Bindings/CSharp/CppSharp.Parser.CSharp.csproj#L9

[konistehrad]

So the state of play:

• I don't think an ARM64 host is building LLVM. I'm still having to build it locally to work.
• My Debian VM fails to complete the LLVM build, so I'm having a hard time testing ARM64 on Linux.
• Mac gets segmentation faults during the final build step. Looks like one of the bindings might be incorrect somehow?

Failure logs:

Build FAILED.

       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/VTables/VTables.Gen.csproj" (default target) (26) ->
       (PostBuildEvent target) -> 
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/VTables.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/VTables/VTables.Gen.csproj]


       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/CLI/CLI.Gen.csproj" (default target) (10) ->
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/CLI.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/CLI/CLI.Gen.csproj]


       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/Common/Common.Gen.csproj" (default target) (16) ->
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/Common.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/Common/Common.Gen.csproj]


       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/StandardLib/StandardLib.Gen.csproj" (default target) (25) ->
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/StandardLib.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/StandardLib/StandardLib.Gen.csproj]


       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/NamespacesDerived/NamespacesDerived.Gen.csproj" (default target) (21) ->
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/NamespacesDerived.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/NamespacesDerived/NamespacesDerived.Gen.csproj]


       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/Encodings/Encodings.Gen.csproj" (default target) (19) ->
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/Encodings.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/Encodings/Encodings.Gen.csproj]


       "/Users/conrad/Documents/CppSharp/CppSharp.sln" (default target) (1) ->
       "/Users/conrad/Documents/CppSharp/tests/dotnet/CSharp/CSharp.Gen.csproj" (default target) (13) ->
         /usr/local/share/dotnet/sdk/8.0.101/Microsoft.Common.CurrentVersion.targets(5809,5): error MSB3073: The command "dotnet "/Users/conrad/Documents/CppSharp/bin/Debug_arm64/CSharp.Gen.dll"" exited with code 139. [/Users/conrad/Documents/CppSharp/tests/dotnet/CSharp/CSharp.Gen.csproj]

    0 Warning(s)
    7 Error(s)

[tritao]

• I don't think an ARM64 host is building LLVM. I'm still having to build it locally to work.

Do you mean on CI? From what I am seeing on the logs, both Mac and Linux seem to be compiling and linking fine. They only fail during runtime when trying to generate the test bindings.

So I would think both the LLVM builds and CppSharp builds are using ARM64 hosts on CI.

[konistehrad]

That's interesting, because attempting to use the generated llvm-6eb36a-linux-arm64-gcc-11-Release yields a bunch of these when attempting to build on Linux ARM64:

/usr/bin/ld: skipping incompatible ../../llvm/llvm-6eb36a-linux-arm64-gcc-11-Release/build/lib/libLLVMDebugInfoPDB.a when searching for -lLLVMDebugInfoPDB /usr/bin/ld: cannot find -lLLVMDebugInfoPDB: No such file or directory

Archive files are a little harder to inspect than shared libs, but I'm used to seeing this error when the generated archive contains the wrong architecture.

EDIT: Indeed, it's hard to tell from the LLVM CI build logs, but here's the relevant line where it indicates it's building on x64 for x64: https://github.com/mono/CppSharp/actions/runs/7760243198/job/21166046377#step:5:221

I assume this means we have to update LLVM.lua to target the platform passed on command line to build.sh. That'll fix the cross compile problem, but not the CI testing problem.

[tritao]

That's interesting, because attempting to use the generated llvm-6eb36a-linux-arm64-gcc-11-Release yields a bunch of these when attempting to build on Linux ARM64:

/usr/bin/ld: skipping incompatible ../../llvm/llvm-6eb36a-linux-arm64-gcc-11-Release/build/lib/libLLVMDebugInfoPDB.a when searching for -lLLVMDebugInfoPDB /usr/bin/ld: cannot find -lLLVMDebugInfoPDB: No such file or directory

Archive files are a little harder to inspect than shared libs, but I'm used to seeing this error when the generated archive contains the wrong architecture.

EDIT: Indeed, it's hard to tell from the LLVM CI build logs, but here's the relevant line where it indicates it's building on x64 for x64: https://github.com/mono/CppSharp/actions/runs/7760243198/job/21166046377#step:5:221

I assume this means we have to update LLVM.lua to target the platform passed on command line to build.sh. That'll fix the cross compile problem, but not the CI testing problem.

Yeah, you are right, missed that.

So, for macOS, looks like Github added new runners, for M1, just a few days ago: https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/

For Linux, looks like they are also working on adding them: https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/

But I am not sure they are available outside private beta yet.

[konistehrad]

Gotcha. Let me cook up some cmake toolchain files to do the LLVM cross compile anyway. That’d save me a ton of time. I’ll make that in a different branch to expedite review and we can kick off a new LLVM job.

[konistehrad]

Actually, while I have you: what's the correct way to determine host architecture in premake? I can't seem to find a way to test to see if we're executing on an x64 host for an arm64 target.

[tritao]

Actually, while I have you: what's the correct way to determine host architecture in premake? I can't seem to find a way to test to see if we're executing on an x64 host for an arm64 target.

For host I think you can use os.host().

And for target architecture:

CppSharp/build/Helpers.lua

Line 92 in 1d1b21f

function target_architecture()

[konistehrad]

Oh, yes, I did find os.host(), unfortunately it doesn't seem to return architecture info. For example, from an M1 Mac in interactive mode:

> print(os.host())
macosx

Not quite enough to do the dirty work I'm afraid. I guess I can use uname on this path since our cross compile paths are limited to Unix-like operating systems.

[konistehrad]

OK, state of play:

• Rosetta context wasn't a good enough test bed for ARM64 cross compiling. I'll need help from someone with access to a real Intel Mac to smooth that out it looks like.
• I updated the Calling Convention from Cdecl to Winapi in std.cs for ARM Mac and it appears to have cleaned some stuff up. It's failing to find some libraries now, but it's progress.
• It looks like we need to add Arm64 to src/CppParser/AST.h, but I don't know yet where else this needs to end up. I'll be working on that as able.

[tritao]

Which libraries is it failing to find?

[konistehrad]

Sorry, I'm not able to continue working on this due to some major life stuff. I'll close it for now. If anyone else wants to pick up the effort just ping me and I'll try to help as able.