CI: build SSR on Windows #352

mgeier · 2023-06-05T22:25:32Z

For now this is just an experiment to check whether it's possible to compile the SSR with MSYS2.

This is already building .exe files (see e.g. artifacts at https://github.com/SoundScapeRenderer/ssr/actions/runs/5191333285), but I don't expect them to actually work. I guess some dependencies will be missing.

For this to be usable, the following steps would have to be completed:

create installer
make sure that all dependencies are static OR include DLLs in package
- including the DLLs seems to make more sense, because they are re-used by all renderer executables
include image files for Qt GUI
include JS/HTML files for browser GUI
include default impulse responses etc.

There are more things to do, but I think those can be done at a later point (in separate PRs), once we know that everything else works:

enable and fix head trackers
select proper config file locations for Windows

Several of those things are already done in #351.

HaHeho · 2023-06-07T00:24:42Z

This is already building .exe files (see e.g. artifacts at https://github.com/SoundScapeRenderer/ssr/actions/runs/5191333285), but I don't expect them to actually work. I guess some dependencies will be missing.

I briefly tried some of these executables after starting JACK. Either by running from the file explorer or via PowerShell, I didn't get any reaction what so ever.

chris-hld · 2023-06-07T07:48:33Z

It might be an issue with QT, it was super hard to get running on windows, as it doesn't complain but just wouldn't start. There are quite a few required extra flags and packages, such as the platform support libs. Could you try to compile without GUI, would that at least give a reaction?

mgeier · 2023-06-07T20:43:12Z

Thanks for testing!

I expected something to be wrong, because the files are too small.

And thanks for the suggestion to disable the GUI, here are the artifacts without GUI: https://github.com/SoundScapeRenderer/ssr/actions/runs/5204312742?pr=352

mgeier · 2023-06-08T10:42:57Z

I might have to use -mwin32 ... at least then the .exe files have a "reasonable" size: https://github.com/SoundScapeRenderer/ssr/actions/runs/5210031866?pr=352

Now each .exe file has a size of 44MB. Now maybe they actually work?

Maybe we can get them smaller, as @chris-hld mentioned in #351 (comment)?

HaHeho · 2023-06-10T00:48:48Z

I might have to use -mwin32 ... at least then the .exe files have a "reasonable" size: https://github.com/SoundScapeRenderer/ssr/actions/runs/5210031866?pr=352

Now each .exe file has a size of 44MB. Now maybe they actually work?

Quickly tried, executed via double click from file explorer, getting the following three errors.

mgeier · 2023-06-10T21:15:29Z

Thanks for testing!

Those error messages look reasonable. I guess those DLLs exist on the build server, but the CI is only uploading the EXE files.

I would hope that the NSIS installer will take care of copying those DLL somehow ... @chris-hld is this true?
If not, we'll have to manually move a few files around.

I have worked a bit more on this and I have implemented a --enable-windows-installer option.
It doesn't yet actually create an installer, but there is a Makefile variable WINDOWS_INSTALLER which contains the desired name of the installer, and the files for the installer are generated in the directory windows-installer when running make nsis.
Within this directory, there is a sub-directory PkgData which contains the data that the SSR needs (GUI images etc.). Currently, SSR_DATA_DIR is set to PkgData but maybe this has to be changed?

All directory names can of course be changed if necessary.

NSIS should already be installed on the MSYS2 system.

@chris-hld Would you like to try adding your installer setup from #351?

chris-hld · 2023-06-13T14:06:26Z

I would hope that the NSIS installer will take care of copying those DLL somehow ... @chris-hld is this true? If not, we'll have to manually move a few files around.

No, I don't think that's possible. NSIS is really just a way of compressing specified binaries to a single installer, plus maybe some extra checks etc.
I used an MXE tool to collect and copy the correct cross-compiled dependencies with
../mxe/tools/copydlldeps.sh --indir ./ssr-win-bin --destdir ./ssr-win-bin/deps --recursivesrcdir ../mxe/usr/x86_64-w64-mingw32.shared/ --srcdir ./_build --copy --enforcedir ./_build/*.dll --enforcedir ../mxe/usr/x86_64-w64-mingw32.shared/qt5/plugins/platforms/

There seems to be an alternative for MSYS2
msys2/MINGW-packages#5204
What worries me a bit is that it is mentioned that MSYS2 is not intended to be portable, so compiling on CI is pointless?

mgeier · 2023-06-13T16:44:07Z

What worries me a bit is that it is mentioned that MSYS2 is not intended to be portable, so compiling on CI is pointless?

I'm not sure if that's true.

I don't really know any of this, but I do know that we are using MSYS2 to compile our Pd externals and they seem to work fine, and they don't have any strange MSYS2 dependencies.

Their dependencies are collected here:

ssr/.github/workflows/pd-externals.yml

Line 127 in 4d9981d

sh pd-lib-builder-iem-ci/pd-lib-builder/localdeps.win.sh build/ssr_*.dll

... which uses this: https://git.iem.at/pd/iem-ci/-/blob/main/localdeps/localdeps.win.sh
... which in turn uses ntldd

I don't understand the comment that suggests using MINGW64 (msys2/MINGW-packages#5204 (comment)), because this somehow seems to be already used within MSYS2.

Although nowadays using UCRT64 seems to be preferred: https://github.com/msys2/setup-msys2#msystem (that's why I'm using it here, but I can switch to MINGW64, if that turns out to work better).

I'll try if I can get the necessary files from ldd as suggested here: msys2/MINGW-packages#5204 (comment)

chris-hld · 2023-06-14T09:54:34Z

Ok, that might be worth a try!
I just plugged in the msys2 compiled exe into the folder containing all the cross-compiled dependencies I collected using MXE and it seems it only complained about missing libfmt for me. So there might be some msys2 specifics, but maybe it's not a lot.

mgeier · 2023-06-14T17:05:52Z

In 637f99e, I have used ldd to find the DLL dependencies, and they are now included in the artifact: https://github.com/SoundScapeRenderer/ssr/actions/runs/5269790198

This includes libfmt.dll but there might still be missing DLLs. Can you please check if it works?

HaHeho · 2023-06-14T23:51:15Z

This includes libfmt.dll but there might still be missing DLLs. Can you please check if it works?

I tried the following two executables. I had to run them via PowerShell, since running them via double click immediately closes the opened terminal without a chance to see the error message. This behavior should be improved somehow, so that a naive Windows user is not entirely lost with what might be going wrong.

PS > & .\ssr-brs.exe
       ___
      /  ___
  ___/  /  ___
    ___/  /    SSR (SoundScape Renderer) 0.6.0-27-gbc9a132
         /     renderer type: BrsRenderer

Website: <http://spatialaudio.net/ssr/>
Contact: <ssr@spatialaudio.net>

Copyright ┬⌐ 2016 Division of Applied Acoustics, Chalmers University of Technology
Copyright ┬⌐ 2014 Institut f├╝r Nachrichtentechnik, Universit├ñt Rostock
Copyright ┬⌐ 2012 Quality & Usability Lab, Telekom Innovation Laboratories, TU Berlin

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
qt.qpa.plugin: Could not find the Qt platform plugin "windows" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

PS > & .\ssr-binaural.exe
       ___
      /  ___
  ___/  /  ___
    ___/  /    SSR (SoundScape Renderer) 0.6.0-27-gbc9a132
         /     renderer type: BinauralRenderer

Website: <http://spatialaudio.net/ssr/>
Contact: <ssr@spatialaudio.net>

Copyright ┬⌐ 2016 Division of Applied Acoustics, Chalmers University of Technology
Copyright ┬⌐ 2014 Institut f├╝r Nachrichtentechnik, Universit├ñt Rostock
Copyright ┬⌐ 2012 Quality & Usability Lab, Telekom Innovation Laboratories, TU Berlin

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Error loading HRIR file: apf::load_sndfile(): "PkgData/default_hrirs.wav" couldn't be loaded!

For the latter, there is a problem with the path of the resource files. They are located in ../PkgData/ relative to bin/ssr-binaural.exe. By moving the contents of bin/ one directory up, I get a similar error to ssr-brs.exe above.

PS > & .\ssr-binaural.exe
       ___
      /  ___
  ___/  /  ___
    ___/  /    SSR (SoundScape Renderer) 0.6.0-27-gbc9a132
         /     renderer type: BinauralRenderer

Website: <http://spatialaudio.net/ssr/>
Contact: <ssr@spatialaudio.net>

Copyright ┬⌐ 2016 Division of Applied Acoustics, Chalmers University of Technology
Copyright ┬⌐ 2014 Institut f├╝r Nachrichtentechnik, Universit├ñt Rostock
Copyright ┬⌐ 2012 Quality & Usability Lab, Telekom Innovation Laboratories, TU Berlin

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
qt.qpa.plugin: Could not find the Qt platform plugin "windows" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

umlaeute · 2023-06-15T10:59:47Z

just to chime in: in my experience, the ntldd (resp ldd) method words nicely for gathering all the dylib dependencies that are encoded in the import section of an executable - because that is what ntldd outputs.

if some binary loads optional functionality via plugins, this method is bound to fail, that's why it doesn't include Qt platform plugins or similar.

mgeier · 2023-06-15T15:49:27Z

Great, it looks like we're making progress!

I have used the windeployqt tool in 2d258b7, which should take care of the Qt platform plugin.

And I've used ../PkgData in 7e09012, but I don't know if the path should use a slash or a backslash. And I don't know whether the directory structure makes sense for the installer anyway. We can also put everything into the same folder ... or whatever. Just let me know which directory names I should choose.

The latest artifact should be available shortly at https://github.com/SoundScapeRenderer/ssr/actions/runs/5280859728?pr=352

HaHeho · 2023-06-16T01:26:41Z

The latest artifact should be available shortly at https://github.com/SoundScapeRenderer/ssr/actions/runs/5280859728?pr=352

The renderer and GUI now opens via double click in file explorer. YAAY.

Loading a config file by executing e.g. .\ssr-binaural.exe ..\Scenes\live_input.asd in PowerShell worked as well. NICE.

Even rendering a complex scene file with multiple long BRIRs in ssr-brs.exe worked without any noticeable artifacts at a JACK block size of 128 samples. AWESOME.

mgeier · 2023-06-17T17:06:55Z

Thanks @HaHeho for testing, this is great news!

I have stripped the .exe files, which makes their size much more manageable (1.5 MB instead of 44 MB).

Now everything together is about 90 MB (about 50 MB compressed), which is still quite big, but I guess it's OK for now.

The latest artifacts are here: https://github.com/SoundScapeRenderer/ssr/actions/runs/5299076783/jobs/9591819774?pr=352

I hope they still work.

I'm curious whether the browser GUI works as well ... @HaHeho can you please try this (https://ssr.readthedocs.io/en/latest/browser-gui.html) when you have some time?

Now the main thing that's missing is the installer.
@chris-hld would you like to tackle this?

I would like to merge #155 soon and then make a new SSR release.

If we have an experimental Windows installer already at that point, we can include it, otherwise we can make another release whenever the Windows installer is ready.

I had to run them via PowerShell, since running them via double click immediately closes the opened terminal without a chance to see the error message. This behavior should be improved somehow, so that a naive Windows user is not entirely lost with what might be going wrong.

I don't know what's the typical solution in this case, but maybe we can configure the installer so that it keeps the terminal open after the SSR is finished?

Or we create a batch file that starts the SSR and waits for a key press afterwards?

HaHeho · 2023-06-18T01:52:38Z

The latest artifacts are here: https://github.com/SoundScapeRenderer/ssr/actions/runs/5299076783/jobs/9591819774?pr=352

I hope they still work.

I tried the same example as yesterday. As far as I can tell, there was no difference, so they seem to be working fine.

I'm curious whether the browser GUI works as well ... @HaHeho can you please try this (https://ssr.readthedocs.io/en/latest/browser-gui.html) when you have some time?

I ran .\ssr-binaural.exe --websocket-server -vvv (and other ssr-variants), and always end up with the same result. The renderer starts just fine, but dies as soon http://localhost:9422/ is accessed via the browser. There is no indication of an error in the terminal log. The last lines are from the initialization.

Starting WebSocket server with port 9422
Floating control panel is used, scene menu will not be shown.

mgeier · 2023-06-18T12:01:26Z

Thanks for testing!

It looks like the problem is with serving the files via HTTP, before even trying to connect via WebSocket.

I guess http://localhost:9422/test will give the same behavior.

It is unfortunate that serving the files doesn't work, but I wouldn't call it a blocker if the WebSocket connection itself works.

Can you please try if it works if you open PkgData/websocket_resources/index.html directly in a browser.
Or if you open ssr-test-client.html and click "connect"?

Also, for completeness, it would be interesting to check the FUDI server as well: https://ssr.readthedocs.io/en/latest/network-fudi.html

HaHeho · 2023-06-18T15:23:41Z

I guess http://localhost:9422/test will give the same behavior.

Yes, it gave the identical behavior to described above.

Can you please try if it works if you open PkgData/websocket_resources/index.html directly in a browser.

Same behaviour, renderer dies without comments.

Or if you open ssr-test-client.html and click "connect"?

Same here. After clicking "connect" it shows "WebSocket connection closed. Code: 1006" in the browser.

Also, for completeness, it would be interesting to check the FUDI server as well: https://ssr.readthedocs.io/en/latest/network-fudi.html

A quick test with muting and muting a couple of sources as well as setting binaural head rotation and rotation offset, was working as expected.

mgeier · 2023-06-23T20:16:27Z

Thanks for testing @HaHeho!

It looks like both HTTP and WebSocket connections lead to the same error.

I have no clue what to do here ... I hope somebody has an idea how to debug this.

It's great that the FUDI interface works, though!

Both interfaces are based on ASIO, so the socket connections should work fine in general.

@chris-hld Did you try this on your MXE-compiled version?
I guess the 3D GUI is not built, but the WebSocket connection seems to be enabled by default?

mgeier · 2023-07-08T08:15:01Z

I have re-based this to the latest SSR release and have added the new ASDF library.

Artifacts are here: https://github.com/SoundScapeRenderer/ssr/actions/runs/5490817426?pr=352

chris-hld · 2023-07-17T10:28:35Z

Thanks for testing @HaHeho!

It looks like both HTTP and WebSocket connections lead to the same error.

I have no clue what to do here ... I hope somebody has an idea how to debug this.

It's great that the FUDI interface works, though!

Both interfaces are based on ASIO, so the socket connections should work fine in general.

@chris-hld Did you try this on your MXE-compiled version? I guess the 3D GUI is not built, but the WebSocket connection seems to be enabled by default?

I don't really have access to a windows machine at the moment, but I thought I tried at some point. It seems that I had to manually set "D_WEBSOCKETPP_CPP11_THREAD_", and I referenced an issue in my CMAKE.
I will try to find a machine and run some more detailed tests on it

chris-hld · 2023-08-07T13:25:39Z

@mgeier The MXE version websocket works as intended.

mgeier · 2023-09-26T19:56:37Z

I have installed VirtualBox and tested this with a Windows 11 image.

Now an installer is created with CI: https://github.com/SoundScapeRenderer/ssr/actions/runs/6317485071?pr=352

Installing and uninstalling works smoothly, and I can start the SSR via the start menu links.
However, there are Qt-related error messages when trying to open a scene.

I could also reproduce the problem with the WebSocket interface: on any connection attempt the SSR crashes without any error message.

@chris-hld

It seems that I had to manually set "D_WEBSOCKETPP_CPP11_THREAD_"

Yes, thanks, I have also found that by reading the source code. Without it, it wouldn't compile.

The MXE version websocket works as intended.

It's good to know that it works when cross-compiling, now we only have to find out how to make it work when compiling with MSYS2.

mgeier mentioned this pull request Jun 5, 2023

SSR is ready for Windows cross-compiling #351

Open

3 tasks

mgeier force-pushed the ci-win branch 2 times, most recently from e945500 to b400b99 Compare June 6, 2023 19:33

mgeier mentioned this pull request Jun 7, 2023

configure.ac: add $EXEEXT #355

Merged

mgeier force-pushed the ci-win branch from b400b99 to 9cf257a Compare June 7, 2023 20:11

mgeier force-pushed the ci-win branch from 9cf257a to 0117487 Compare June 8, 2023 10:13

mgeier force-pushed the ci-win branch from 0117487 to 2bc4ecf Compare June 10, 2023 21:07

mgeier force-pushed the ci-win branch from 2bc4ecf to a88b92c Compare June 11, 2023 08:29

mgeier force-pushed the ci-win branch from 85745dd to f0f8fd1 Compare July 7, 2023 21:46

mgeier added 9 commits September 26, 2023 21:34

Implement --enable-windows-installer based on --enable-app-bundle

61dd385

CI: build SSR on Windows

cad290d

Copy DLL dependencies

e3e8644

Use windeployqt

8ea30f2

PkgData -> ../PkgData

4fdd374

Strip .exe files

8e666e6

copy dependencies in Makefile

3de1a23

--enable-dynamic-asdf

e3c57ca

Add first version of a Windows installer

9d9a23f

mgeier force-pushed the ci-win branch from 2bd12fc to 9d9a23f Compare September 26, 2023 19:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: build SSR on Windows #352

CI: build SSR on Windows #352

mgeier commented Jun 5, 2023 •

edited

Loading

HaHeho commented Jun 7, 2023

chris-hld commented Jun 7, 2023 •

edited

Loading

mgeier commented Jun 7, 2023

mgeier commented Jun 8, 2023

HaHeho commented Jun 10, 2023

mgeier commented Jun 10, 2023

chris-hld commented Jun 13, 2023

mgeier commented Jun 13, 2023

chris-hld commented Jun 14, 2023

mgeier commented Jun 14, 2023

HaHeho commented Jun 14, 2023

umlaeute commented Jun 15, 2023

mgeier commented Jun 15, 2023

HaHeho commented Jun 16, 2023

mgeier commented Jun 17, 2023

HaHeho commented Jun 18, 2023

mgeier commented Jun 18, 2023

HaHeho commented Jun 18, 2023

mgeier commented Jun 23, 2023

mgeier commented Jul 8, 2023

chris-hld commented Jul 17, 2023

chris-hld commented Aug 7, 2023

mgeier commented Sep 26, 2023

CI: build SSR on Windows #352

Are you sure you want to change the base?

CI: build SSR on Windows #352

Conversation

mgeier commented Jun 5, 2023 • edited Loading

HaHeho commented Jun 7, 2023

chris-hld commented Jun 7, 2023 • edited Loading

mgeier commented Jun 7, 2023

mgeier commented Jun 8, 2023

HaHeho commented Jun 10, 2023

mgeier commented Jun 10, 2023

chris-hld commented Jun 13, 2023

mgeier commented Jun 13, 2023

chris-hld commented Jun 14, 2023

mgeier commented Jun 14, 2023

HaHeho commented Jun 14, 2023

umlaeute commented Jun 15, 2023

mgeier commented Jun 15, 2023

HaHeho commented Jun 16, 2023

mgeier commented Jun 17, 2023

HaHeho commented Jun 18, 2023

mgeier commented Jun 18, 2023

HaHeho commented Jun 18, 2023

mgeier commented Jun 23, 2023

mgeier commented Jul 8, 2023

chris-hld commented Jul 17, 2023

chris-hld commented Aug 7, 2023

mgeier commented Sep 26, 2023

mgeier commented Jun 5, 2023 •

edited

Loading

chris-hld commented Jun 7, 2023 •

edited

Loading