Why is there no libJOCL_2_0_4-linux-arm64.so? #48
Comments
|
Nobody provided it. I can not (easily or reasonably) build the binaries for all possible combinations of operating systems and architectures. You can compile the native library on your own, as described at https://github.com/gpu/JOCL#building-the-native-libraries , and put the native library into your project root directory (it should be picked up from there, usually). If someone is willing to contribute pre-built binaries for other architectures, then they can be integrated into future releases as well. (There is no timeline for specific updates for the near future, but in doubt, I could create a Maven release with the additional library, if someone provided it) |
|
I will try and build it and contribute it back. Thank you for providing aarch anyway that one was very useful. |
|
I've deployed an arm64 VM and installed the relevant tools, I should be able to supply the lib back to the project this week. @gpu what version of the JDK should I build it against? |
|
I don't know whether that makes any difference. I guess it's just OpenJDK? Maybe @blueberry can give a hint here. |
|
I don't know. I've never used arm64, nor do I have a computer with that architecture. |
Do you have a JDK installed right now (and not only a JRE)? (I think that CMake should be able to figure out these paths automatically, but that might not be the case here, depending on where/how exactly it was installed...) |
|
@gpu @blueberry Its a fresh VM, I didn't want to have any chance of contamination for a library that other people may rely on. I can install any jdk version really, even jdk 8 is available on arm64. The question is just about the java version that the other libraries were compiled against. If you used the header files from jdk 10, for example, for all the other libs - I would install jdk10. If it really doesn't matter, I will install jdk17. |
|
And to be clear, I would be using openjdk for versions 10 or above, and oracle jdk for version 9 and below. But the header files are essentially part of the specification, I don't think the vendor of the jdk is relevant - only the version. |
|
The version may be a relevant point here. Usually, in terms of compatibility, it's good to use "the lowest version that works". To be honest, I never thought about where and how the JDK version comes into play for JOCL (that's the good thing with Java - it usually "just works"...). But I'd like to avoid a situation where someone has Java 10, and JOCL bails out due to some dependency to a newer Java version. If it works with JDK 8 (which I'm still using, by the way), then it could/should be that. |
|
I built it on arm using jdk 8. @gpu @blueberry but for some reason its labeled as x86_64. is this a known issue? |
|
The "x86_64" part is only part of the file name, used for disambiguating (the specific combination of Linux+Arm is not yet covered at Line 44 in fefa66d So it should be OK to just rename that file. It would be important to know what returns on such a system, to make sure that the name matches what is computed at JOCL/src/main/java/org/jocl/LibUtils.java Line 580 in fefa66d |
|
It emits the same value as shown in my response above: I assume I can fork the repo where this is located, and open a pull request? Where are the native libs located? The correct name should be: is that right? |
|
I have uploaded the library to dropbox so that you can include it in a future release if you are able: https://www.dropbox.com/s/fzm9mc83ynfchs1/libJOCL_2_0_4-linux-arm64.so?dl=0 |
|
Thanks @ashesfall - there wasn't a release planned, but I'll schedule some time for an update, and integrade the new binary with that. |
|
@gpu thank you. this repository does not appear to be participating in github sponsors. If there is another way to support the project let me know. |
|
@ashesfall thanks for your work! I have tried using your lib but I get this error: I'm on ubuntu 20.04 and tried to update but I only get version 2.31 I have tried to build by myself without success with these build-instructions
CMakeOutput.log Can you give me a hint how to build it correctly? |
|
@aspecialkey Is java working at all on your machine? if so, which version of java? |
|
|
Solved the problem for me. |
Nobody is willing to pay for open source software. (There are reasons why there hasn't been sooo much active development around JOCL for a while now, e.g. there are no updates to support newer OpenCL versions...)
The problem of GLIBC compatibility also appeared in one of my other projects. In general, it would be good to have the binaries compiled with the lowest GLIBC version that works. Otherwise, people might have GLIBC version 2.39999999 and receive an error message because the binary was built against 2.34. So @aspecialkey If you managed to compile the binary, then maybe you could post it here. Then @ashesfall could try it out (and it should almost certainly work), and maybe we can create a new JOCL release that runs on a broader range of target OSes (i.e. ones with older GLIBC versions) |
|
@gpu I'm pretty much committed to a version of GLIBC via the OS. Do you have a recommendation for a linux distro and version that would have the "lowest" usable GLIBC with it? |
|
I'm not deeply familiar with some aspects of developing on Linux that may be relevant here. And I don't know what the "lowest" version is that will work, but ... I think that nothing relevant has happened here in the past ~8 years, so I think it roughly depends on the question: Which is the lowest Linux/GLIBC version on which an OpenCL 2.0 SDK can be installed? But @ashesfall you probably don't want to downgrade your OS to an ancient version, just to compile this lib. Possible solutions could be:
|
|
My library I have compiled on Ubuntu LTS 20.04 against Java 11. Next week I could recompile against Java 8 on Ubuntu LTS 18.04 if I remember correctly GLIBC version 2.27. That would bring greater compatibility I think. or if @ashesfall is faster, of course you are welcome to do that too ;) |
|
@gpu I'm using cloud VMs, I don't even own a physical ARM device that isn't made by Apple @aspecialkey I'm unlikely to get to it until early next week, but I will eventually do it as my own library depends on this one and really needs to run on Amazon's ARM machines. |
|
@ashesfall I considered setting up some build infrastructure (basically playing with some GitHub action for building on "all OSes and architectures", and trying to grab the resulting native libraries). This might be "easy" in a technical sense, like just adding the right So... if @ashesfall or @aspecialkey provides a library (and maybe the other one checks whether it "also works on other machines"), I can schedule a version |
|
@gpu it would be pretty easy to do this with github actions, I could probably help set it up. In the meantime though I'm just going to rebuild the library on an older OS (older glibc) for wider compatibility. I didn't find the time to do it over last weekend, so it probably wont be until this weekend that I find the time. |
That would be cool. Setting up the action itself would probably be relatively easy. What I currently don't know: This "action" will run some command-line CMake build, and generate some |
|
@gpu I finally got around to building it with GLIBC 2.27 for greater compatibility, and you can download it here https://www.dropbox.com/s/ojiovlc6yi7qz3g/libJOCL_2_0_4-linux-arm64.so?dl=0 To do this, I used a method that would be directly portable to github actions to see how straightforward that would be. I created this Dockerfile: Then I run: To retrieve the library. If I find some time during May to do it, I'll create a github actions configuration that does the same thing and submit a merge request for the repo. |
|
Thanks @ashesfall . I'll schedule a new release, and try to find some time to look at the docker approach. (When I previously compiled binaries in VMs, I usually hesitated to release them, because I couldn't really test them, but at some point, I just have to assume that (and create a patch verson if necessary...)) |
|
@gpu yea, the community ultimately has to validate to be completely sure nothing is broken, but a test suite can be created to run in the docker container as well. Since I believe the custom github runners (which I think you would need for some of the arch variants) are still in beta, you may want to apply to join the beta now if you plan on doing something like this eventually. |
However, JOCL 2.0.5 should be on its way into Maven Central. |
|
In any case: Thanks @ashesfall for your support here! |
|
@gpu no problem. this project is critical to my own (https://github.com/almostrealism/common/blob/master/README.md) |
|
That project looks interesting - but huge. I only skimmed over some parts of it (and could try to allocate more time for wrapping my head around all that, but ... there are too many projects out there for which I would do that, iff I had infinite time...). The idea of generalizing certain common mathematical operations and adding convenient abstraction layers (preferably ones that can transparently be executed on the CPU or GPU) is not entirely new, but I don't have an overview of all the libraries that already exist out there. Some bragging ahead.... During my quick scrolling, I saw Even just having a bunch of predefined kernels to operate on vector data can be useful (similar to https://github.com/jcuda/jcuda-vec ), but this could be generalized by offering certain structures in a "CPU form" and a "GPU form" (similar to https://github.com/jcuda/jcuda-matrix-utils). (Yes, that's yet another one of my identities). One goal of such abstractions could be to apply generic operations on bulk data, like in pseucodode.... The obvious next generalization would be to support ... arbitrary operations there. This might be similar to what you did in the https://github.com/almostrealism/common/tree/master/code package. Applied to the pseudocode, this would be with some basic expression parser that reads that expression string. And... this is not even too difficult: There are a bunch of parsers out there that can generate an AST, and that AST can be converted into OpenCL code, and that can be compiled into a kernel at runtime. I did this once, for Groovy, at http://jocl.org/GroovyGPU/ - but it would also be possible for Java. That's essentially what https://github.com/Syncleus/aparapi is doing (even though that project went through quite some hiccups in the past...) When this is rather supposed to run on the CPU, I have done some work there as well: In my (private!) project https://github.com/javagl/Data there already are packages for generic data set representations, general I/O (using Weka, JSON, CSV, ...), general matrix operations, and a package called (e.g. for computing an object that is closest to another object, based on some arbitrary distance function). All that internally takes care of the efficient, parallel-stream based execution of all these functions. And finally... I've seen that But again: There are too many things too look at.... |
|
@gpu Your javagl space looks interesting, do you have working demos of hardware accelerated 3D from Java? I was using JOGL for the UI in my audio generation (built on top of the library I sent) app, but I ended up switching to the much less flexible (but more reliable on different systems) built-in JavaFX 3d support. I'd like to switch back, but I haven't kept up with where JOGL is going (if anywhere). As for Tensor: it, like most of the library, is not hardware accelerated. It can support ND collections of arbitrary objects, but it must be converted to PackedCollection for use with hardware acceleration using You are right that the I'm mainly using these tools for generating audio, describing the space of possible music arrangements using a data structure called Thanks for your reply. Sorry it took me so long to respond. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've switched to arm64 to support Apple, and its awesome to see libJOCL_2_0_4-apple-arm64.dylib in the jar, but what about libJOCL_2_0_4-linux-arm64.so?
I need this to work on linux also, and I don't see why it wouldn't - save for this being missing.
The text was updated successfully, but these errors were encountered: