Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for 7 compression plugins #35

Merged
merged 3 commits into from
Feb 6, 2024
Merged

Add support for 7 compression plugins #35

merged 3 commits into from
Feb 6, 2024

Conversation

axelboc
Copy link
Contributor

@axelboc axelboc commented Dec 20, 2023

Took me a while to figure out how to fetch the plugin files from the webview, but I got there in the end. Unfortunately, I'm hitting the following issue with some of the larger plugins (LZ4, ZStandard, etc.):

hdf5_util.js:8 RangeError: WebAssembly.Compile is disallowed on the main thread, if the buffer size is larger than 4KB.
Use WebAssembly.compile, or compile on a worker thread.

The webview fetches the plugin files correctly, and they get written to the Emscripten file system as expected. The problem seems to occur when h5wasm reads the compressed dataset and HDF5 tries to load the appropriate plugin dynamically.

@bmaranville I assume there's no other workaround than to move @h5web/h5wasm to a worker, right?

@bmaranville
Copy link

This was a limitation of Chrome and Chrome-based browsers (including headless, I'm sure) until recently. The limitation was removed months ago (just before the HDF5 User Group meeting - I talked about it there). Simplest solution is just to upgrade to a more recent browser version. The issue doesn't seem to affect other engines (like those in Safari or Firefox)

@axelboc
Copy link
Contributor Author

axelboc commented Dec 20, 2023

Right, hmm this sucks then, because I'm on the latest version of VS Code... Maybe VS Code or Electron enforces this limitation internally on purpose to avoid blocking the main thread. 🤔

@bmaranville
Copy link

bmaranville commented Dec 20, 2023

I'm guessing it means they're still on a slightly older JS engine - eventually the problem will go away (maybe?)

@bmaranville
Copy link

If you are going to move it to a worker, it seems like you could reuse the h5grove api definitions and write a worker that provides those endpoints. I don't imagine that would be too much work, and could be useful in the browser too.

@bmaranville
Copy link

Have you looked into preloading the plugins, as described here? https://emscripten.org/docs/porting/files/packaging_files.html#preloading-files

With version of Chromium bundled with Electron 27, the compression plugin files are successfully compiled to WebAssembly on the main thread when loaded.

https://code.visualstudio.com/updates/v1_86#_electron-27-update
@axelboc axelboc marked this pull request as ready for review February 5, 2024 15:32
@axelboc
Copy link
Contributor Author

axelboc commented Feb 5, 2024

Well... seems that patience pays! 😂 With the upgrade to Electron 27 in VS Code 1.86.0, the error is gone and the plugins load successfully. 🎉

I'll keep the goal of moving to a worker on the back burner, as this is still the better approach in the long term.

package.json Show resolved Hide resolved
src/H5WebViewer.ts Show resolved Hide resolved
src/H5WebViewer.ts Show resolved Hide resolved
@axelboc axelboc merged commit 53805b0 into main Feb 6, 2024
6 checks passed
@axelboc axelboc deleted the h5wasm-plugins branch February 6, 2024 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants