Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRYPTO-172: Add support for Linux-riscv64 #264

Merged
merged 23 commits into from
Nov 4, 2023

Conversation

luhenry
Copy link
Contributor

@luhenry luhenry commented Oct 30, 2023

Java supports RISC-V since version 19. Given Apache Common Crypto is a commonly used library in the Java ecosystem, it's important for it to support RISC-V as well.

This patch adds the linux-riscv64 target similarly to linux-aarch64.

I've verified it builds with docker compose -f src/docker/docker-compose.yaml run crypto src/docker/build.sh.

Java supports RISC-V since version 19. Given Apache Common Crypto is a
commonly used library in the Java ecosystem, it's important for it to
support RISC-V as well.

This patch adds the linux-riscv64 target similarly to linux-aarch64
Copy link
Contributor

@sebbASF sebbASF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need to ensure that the GH Actions include some tests for the architecture if possible.

src/docker/Dockerfile Outdated Show resolved Hide resolved
@luhenry
Copy link
Contributor Author

luhenry commented Oct 30, 2023

@sebbASF I couldn't find how the test is run on linux-aarch64. Where could I find it?

@sebbASF
Copy link
Contributor

sebbASF commented Oct 30, 2023

I don't know, that's why I asked. I assume there is some GH documentation on which architectures are available.
If risc64 is not available as a test host, then this needs to be documented, as it means the combination has not been tested.

Run with docker compose -f src/docker/docker-compose.yaml run crypto-riscv64-test
```
$> docker compose -f src/docker/docker-compose.yaml run crypto-riscv64-test
WARN[0000] Found orphan containers ([docker-cross-compile-riscv64-build-run-397c7aebdada docker-cross-compile-riscv64-runtime-setup-1 docker-cross-compile-riscv64-build-run-93a982a05cdc docker-cross-compile-riscv64-build-run-7dc6a41f1d5b docker-cross-compile-riscv64-build-run-644e853cbf55 docker-cross-compile-riscv64-build-run-3b09454dc280]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Running 1/1
 ! crypto-riscv64-test Warning                                                                                 2.0s
[+] Building 303.7s (9/9) FINISHED                                                docker-container:festive_mccarthy
 => [crypto-riscv64-test internal] load build definition from Dockerfile.riscv64                               0.0s
 => => transferring dockerfile: 1.97kB                                                                         0.0s
 => [crypto-riscv64-test internal] load metadata for docker.io/riscv64/ubuntu:20.04                            0.7s
 => [crypto-riscv64-test internal] load .dockerignore                                                          0.0s
 => => transferring context: 2B                                                                                0.0s
 => CACHED [crypto-riscv64-test 1/4] FROM docker.io/riscv64/ubuntu:20.04@sha256:c0e96f609e5f128bdb6d24d21d3bf  0.0s
 => => resolve docker.io/riscv64/ubuntu:20.04@sha256:c0e96f609e5f128bdb6d24d21d3bf0a9af6f17234976fa574dad7146  0.0s
 => [crypto-riscv64-test 2/4] RUN apt-get update && apt-get --assume-yes install software-properties-common  274.2s
 => [crypto-riscv64-test 3/4] RUN curl -L https://dlcdn.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-  1.0s
 => [crypto-riscv64-test 4/4] WORKDIR /home/crypto                                                             0.0s
 => [crypto-riscv64-test] exporting to docker image format                                                    27.7s
 => => exporting layers                                                                                       17.4s
 => => exporting manifest sha256:9068927ccfbc46788ec15578541ebc06302b538d4ed75e308b694e1132dd55fd              0.0s
 => => exporting config sha256:27adda314855e324670f04d2cc656b48c19879ccf16b9e4f32f32cd273bb3319                0.0s
 => => sending tarball                                                                                        10.2s
 => [crypto-riscv64-test crypto-riscv64-test] importing to docker                                              9.1s
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: /opt/maven
Java version: 21-ea, vendor: Private Build, runtime: /usr/lib/jvm/java-21-openjdk-riscv64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "6.2.0-35-generic", arch: "riscv64", family: "unix"
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< org.apache.commons:commons-crypto >------------------
[INFO] Building Apache Commons Crypto 1.2.1-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-surefire-plugin:3.1.2:test (default-cli) @ commons-crypto ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.commons.crypto.CryptoTest
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/commons-crypto-1e590661-aa26-4da1-b5c2-8d721ca08d18-libcommons-crypto.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.095 s -- in org.apache.commons.crypto.CryptoTest
[INFO] Running org.apache.commons.crypto.OsInfoTest
Linux/riscv64
Linux
riscv64
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.663 s -- in org.apache.commons.crypto.OsInfoTest
[INFO] Running org.apache.commons.crypto.utils.UtilsTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.159 s -- in org.apache.commons.crypto.utils.UtilsTest
[INFO] Running org.apache.commons.crypto.utils.EnumTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.033 s -- in org.apache.commons.crypto.utils.EnumTest
[INFO] Running org.apache.commons.crypto.jna.CtrCryptoJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.60 s -- in org.apache.commons.crypto.jna.CtrCryptoJnaStreamTest
[INFO] Running org.apache.commons.crypto.jna.CtrNoPaddingCipherJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.532 s -- in org.apache.commons.crypto.jna.CtrNoPaddingCipherJnaStreamTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslJnaTest
Apache Commons Crypto OpenSslJna: enabled = true, version = 0x1010106F
OpenSSLVersion(0): OpenSSL 1.1.1f  31 Mar 2020
OpenSSLVersion(1): compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-zF9Nk8/openssl-1.1.1f=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OpenSSLVersion(2): built on: Tue Oct 10 09:03:48 2023 UTC
OpenSSLVersion(3): platform: debian-riscv64
OpenSSLVersion(4): OPENSSLDIR: "/usr/lib/ssl"
OpenSSLVersion(5): ENGINESDIR: "/usr/lib/riscv64-linux-gnu/engines-1.1"
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.394 s -- in org.apache.commons.crypto.jna.OpenSslJnaTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslJnaCipherTest
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.447 s -- in org.apache.commons.crypto.jna.OpenSslJnaCipherTest
[INFO] Running org.apache.commons.crypto.jna.CbcNoPaddingCipherJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.448 s -- in org.apache.commons.crypto.jna.CbcNoPaddingCipherJnaStreamTest
[INFO] Running org.apache.commons.crypto.jna.PositionedCryptoInputStreamJnaTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.210 s -- in org.apache.commons.crypto.jna.PositionedCryptoInputStreamJnaTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslNativeJnaTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.300 s -- in org.apache.commons.crypto.jna.OpenSslNativeJnaTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslJnaCryptoRandomTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.24 s -- in org.apache.commons.crypto.jna.OpenSslJnaCryptoRandomTest
[INFO] Running org.apache.commons.crypto.jna.CbcPkcs5PaddingCipherJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.609 s -- in org.apache.commons.crypto.jna.CbcPkcs5PaddingCipherJnaStreamTest
[INFO] Running org.apache.commons.crypto.cipher.CryptoCipherTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.328 s -- in org.apache.commons.crypto.cipher.CryptoCipherTest
[INFO] Running org.apache.commons.crypto.cipher.OpenSslCipherTest
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.684 s -- in org.apache.commons.crypto.cipher.OpenSslCipherTest
[INFO] Running org.apache.commons.crypto.cipher.CryptoCipherFactoryTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.266 s -- in org.apache.commons.crypto.cipher.CryptoCipherFactoryTest
[INFO] Running org.apache.commons.crypto.cipher.OpenSslCommonModeTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 s -- in org.apache.commons.crypto.cipher.OpenSslCommonModeTest
[INFO] Running org.apache.commons.crypto.cipher.GcmCipherTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.431 s -- in org.apache.commons.crypto.cipher.GcmCipherTest
[INFO] Running org.apache.commons.crypto.cipher.JceCipherTest
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.126 s -- in org.apache.commons.crypto.cipher.JceCipherTest
[INFO] Running org.apache.commons.crypto.stream.CbcPkcs5PaddingCipherStreamTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.579 s -- in org.apache.commons.crypto.stream.CbcPkcs5PaddingCipherStreamTest
[INFO] Running org.apache.commons.crypto.stream.output.StreamOutputTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 s -- in org.apache.commons.crypto.stream.output.StreamOutputTest
[INFO] Running org.apache.commons.crypto.stream.CbcNoPaddingCipherStreamTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.528 s -- in org.apache.commons.crypto.stream.CbcNoPaddingCipherStreamTest
[INFO] Running org.apache.commons.crypto.stream.input.ChannelInputTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 s -- in org.apache.commons.crypto.stream.input.ChannelInputTest
[INFO] Running org.apache.commons.crypto.stream.PositionedCryptoInputStreamTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.747 s -- in org.apache.commons.crypto.stream.PositionedCryptoInputStreamTest
[INFO] Running org.apache.commons.crypto.stream.CtrNoPaddingCipherStreamTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 s -- in org.apache.commons.crypto.stream.CtrNoPaddingCipherStreamTest
[INFO] Running org.apache.commons.crypto.stream.CtrCryptoStreamTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.703 s -- in org.apache.commons.crypto.stream.CtrCryptoStreamTest
[INFO] Running org.apache.commons.crypto.random.OpenSslCryptoRandomTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.651 s -- in org.apache.commons.crypto.random.OpenSslCryptoRandomTest
[INFO] Running org.apache.commons.crypto.random.CryptoRandomFactoryTest
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.530 s -- in org.apache.commons.crypto.random.CryptoRandomFactoryTest
[INFO] Running org.apache.commons.crypto.random.JavaCryptoRandomTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.79 s -- in org.apache.commons.crypto.random.JavaCryptoRandomTest
[INFO] Running org.apache.commons.crypto.random.OsCryptoRandomTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.594 s -- in org.apache.commons.crypto.random.OsCryptoRandomTest
[INFO] Running org.apache.commons.crypto.NativeCodeLoaderTest
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 1.206 s -- in org.apache.commons.crypto.NativeCodeLoaderTest
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 141, Failures: 0, Errors: 0, Skipped: 3
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:18 min
[INFO] Finished at: 2023-10-31T13:04:26Z
[INFO] ------------------------------------------------------------------------
```
@luhenry
Copy link
Contributor Author

luhenry commented Oct 31, 2023

I've added a docker container to run the test and ran it successfully:

$> docker compose -f src/docker/docker-compose.yaml run crypto-riscv64-test
WARN[0000] Found orphan containers ([docker-cross-compile-riscv64-build-run-397c7aebdada docker-cross-compile-riscv64-runtime-setup-1 docker-cross-compile-riscv64-build-run-93a982a05cdc docker-cross-compile-riscv64-build-run-7dc6a41f1d5b docker-cross-compile-riscv64-build-run-644e853cbf55 docker-cross-compile-riscv64-build-run-3b09454dc280]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Running 1/1
 ! crypto-riscv64-test Warning                                                                                 2.0s
[+] Building 303.7s (9/9) FINISHED                                                docker-container:festive_mccarthy
 => [crypto-riscv64-test internal] load build definition from Dockerfile.riscv64                               0.0s
 => => transferring dockerfile: 1.97kB                                                                         0.0s
 => [crypto-riscv64-test internal] load metadata for docker.io/riscv64/ubuntu:20.04                            0.7s
 => [crypto-riscv64-test internal] load .dockerignore                                                          0.0s
 => => transferring context: 2B                                                                                0.0s
 => CACHED [crypto-riscv64-test 1/4] FROM docker.io/riscv64/ubuntu:20.04@sha256:c0e96f609e5f128bdb6d24d21d3bf  0.0s
 => => resolve docker.io/riscv64/ubuntu:20.04@sha256:c0e96f609e5f128bdb6d24d21d3bf0a9af6f17234976fa574dad7146  0.0s
 => [crypto-riscv64-test 2/4] RUN apt-get update && apt-get --assume-yes install software-properties-common  274.2s
 => [crypto-riscv64-test 3/4] RUN curl -L https://dlcdn.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-  1.0s
 => [crypto-riscv64-test 4/4] WORKDIR /home/crypto                                                             0.0s
 => [crypto-riscv64-test] exporting to docker image format                                                    27.7s
 => => exporting layers                                                                                       17.4s
 => => exporting manifest sha256:9068927ccfbc46788ec15578541ebc06302b538d4ed75e308b694e1132dd55fd              0.0s
 => => exporting config sha256:27adda314855e324670f04d2cc656b48c19879ccf16b9e4f32f32cd273bb3319                0.0s
 => => sending tarball                                                                                        10.2s
 => [crypto-riscv64-test crypto-riscv64-test] importing to docker                                              9.1s
Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39)
Maven home: /opt/maven
Java version: 21-ea, vendor: Private Build, runtime: /usr/lib/jvm/java-21-openjdk-riscv64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "6.2.0-35-generic", arch: "riscv64", family: "unix"
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< org.apache.commons:commons-crypto >------------------
[INFO] Building Apache Commons Crypto 1.2.1-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-surefire-plugin:3.1.2:test (default-cli) @ commons-crypto ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.commons.crypto.CryptoTest
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/commons-crypto-1e590661-aa26-4da1-b5c2-8d721ca08d18-libcommons-crypto.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.095 s -- in org.apache.commons.crypto.CryptoTest
[INFO] Running org.apache.commons.crypto.OsInfoTest
Linux/riscv64
Linux
riscv64
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.663 s -- in org.apache.commons.crypto.OsInfoTest
[INFO] Running org.apache.commons.crypto.utils.UtilsTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.159 s -- in org.apache.commons.crypto.utils.UtilsTest
[INFO] Running org.apache.commons.crypto.utils.EnumTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.033 s -- in org.apache.commons.crypto.utils.EnumTest
[INFO] Running org.apache.commons.crypto.jna.CtrCryptoJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.60 s -- in org.apache.commons.crypto.jna.CtrCryptoJnaStreamTest
[INFO] Running org.apache.commons.crypto.jna.CtrNoPaddingCipherJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.532 s -- in org.apache.commons.crypto.jna.CtrNoPaddingCipherJnaStreamTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslJnaTest
Apache Commons Crypto OpenSslJna: enabled = true, version = 0x1010106F
OpenSSLVersion(0): OpenSSL 1.1.1f  31 Mar 2020
OpenSSLVersion(1): compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-zF9Nk8/openssl-1.1.1f=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OpenSSLVersion(2): built on: Tue Oct 10 09:03:48 2023 UTC
OpenSSLVersion(3): platform: debian-riscv64
OpenSSLVersion(4): OPENSSLDIR: "/usr/lib/ssl"
OpenSSLVersion(5): ENGINESDIR: "/usr/lib/riscv64-linux-gnu/engines-1.1"
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.394 s -- in org.apache.commons.crypto.jna.OpenSslJnaTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslJnaCipherTest
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.447 s -- in org.apache.commons.crypto.jna.OpenSslJnaCipherTest
[INFO] Running org.apache.commons.crypto.jna.CbcNoPaddingCipherJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.448 s -- in org.apache.commons.crypto.jna.CbcNoPaddingCipherJnaStreamTest
[INFO] Running org.apache.commons.crypto.jna.PositionedCryptoInputStreamJnaTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.210 s -- in org.apache.commons.crypto.jna.PositionedCryptoInputStreamJnaTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslNativeJnaTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.300 s -- in org.apache.commons.crypto.jna.OpenSslNativeJnaTest
[INFO] Running org.apache.commons.crypto.jna.OpenSslJnaCryptoRandomTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.24 s -- in org.apache.commons.crypto.jna.OpenSslJnaCryptoRandomTest
[INFO] Running org.apache.commons.crypto.jna.CbcPkcs5PaddingCipherJnaStreamTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.609 s -- in org.apache.commons.crypto.jna.CbcPkcs5PaddingCipherJnaStreamTest
[INFO] Running org.apache.commons.crypto.cipher.CryptoCipherTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.328 s -- in org.apache.commons.crypto.cipher.CryptoCipherTest
[INFO] Running org.apache.commons.crypto.cipher.OpenSslCipherTest
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.684 s -- in org.apache.commons.crypto.cipher.OpenSslCipherTest
[INFO] Running org.apache.commons.crypto.cipher.CryptoCipherFactoryTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.266 s -- in org.apache.commons.crypto.cipher.CryptoCipherFactoryTest
[INFO] Running org.apache.commons.crypto.cipher.OpenSslCommonModeTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 s -- in org.apache.commons.crypto.cipher.OpenSslCommonModeTest
[INFO] Running org.apache.commons.crypto.cipher.GcmCipherTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.431 s -- in org.apache.commons.crypto.cipher.GcmCipherTest
[INFO] Running org.apache.commons.crypto.cipher.JceCipherTest
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.126 s -- in org.apache.commons.crypto.cipher.JceCipherTest
[INFO] Running org.apache.commons.crypto.stream.CbcPkcs5PaddingCipherStreamTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.579 s -- in org.apache.commons.crypto.stream.CbcPkcs5PaddingCipherStreamTest
[INFO] Running org.apache.commons.crypto.stream.output.StreamOutputTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 s -- in org.apache.commons.crypto.stream.output.StreamOutputTest
[INFO] Running org.apache.commons.crypto.stream.CbcNoPaddingCipherStreamTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.528 s -- in org.apache.commons.crypto.stream.CbcNoPaddingCipherStreamTest
[INFO] Running org.apache.commons.crypto.stream.input.ChannelInputTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 s -- in org.apache.commons.crypto.stream.input.ChannelInputTest
[INFO] Running org.apache.commons.crypto.stream.PositionedCryptoInputStreamTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.747 s -- in org.apache.commons.crypto.stream.PositionedCryptoInputStreamTest
[INFO] Running org.apache.commons.crypto.stream.CtrNoPaddingCipherStreamTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471 s -- in org.apache.commons.crypto.stream.CtrNoPaddingCipherStreamTest
[INFO] Running org.apache.commons.crypto.stream.CtrCryptoStreamTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.703 s -- in org.apache.commons.crypto.stream.CtrCryptoStreamTest
[INFO] Running org.apache.commons.crypto.random.OpenSslCryptoRandomTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.651 s -- in org.apache.commons.crypto.random.OpenSslCryptoRandomTest
[INFO] Running org.apache.commons.crypto.random.CryptoRandomFactoryTest
[INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.530 s -- in org.apache.commons.crypto.random.CryptoRandomFactoryTest
[INFO] Running org.apache.commons.crypto.random.JavaCryptoRandomTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.79 s -- in org.apache.commons.crypto.random.JavaCryptoRandomTest
[INFO] Running org.apache.commons.crypto.random.OsCryptoRandomTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.594 s -- in org.apache.commons.crypto.random.OsCryptoRandomTest
[INFO] Running org.apache.commons.crypto.NativeCodeLoaderTest
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 1.206 s -- in org.apache.commons.crypto.NativeCodeLoaderTest
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 141, Failures: 0, Errors: 0, Skipped: 3
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:18 min
[INFO] Finished at: 2023-10-31T13:04:26Z
[INFO] ------------------------------------------------------------------------

@luhenry
Copy link
Contributor Author

luhenry commented Oct 31, 2023

@sebbASF I've added testing on GitHub Actions for linux-aarch64 and linux-riscv64. You'll note I had to downgrade JNA to 5.12 because of java-native-access/jna#1557.

@garydgregory
Copy link
Member

I would not rollback JNA for ALL configurations just because one has special requirements, just downgrade in that case.

@luhenry
Copy link
Contributor Author

luhenry commented Oct 31, 2023

I would not rollback JNA for ALL configurations just because one has special requirements, just downgrade in that case.

It's a build-time dependency with the commons-crypto package being the same across all platforms. I'm afraid there is not an easy way to specify per-arch dependency in that case. I'll keep monitoring the JNA linux-riscv64 issue and update anything here whenever there is an update (that we merge this PR with the downgrade first, or that we wait to merge that PR)

Copy link
Contributor

@sebbASF sebbASF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see individual review comments

src/docker/Dockerfile Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
src/site/xdoc/index.xml Outdated Show resolved Hide resolved
src/docker/Dockerfile.riscv64 Outdated Show resolved Hide resolved
src/docker/Dockerfile.aarch64 Outdated Show resolved Hide resolved
@sebbASF
Copy link
Contributor

sebbASF commented Oct 31, 2023

@sebbASF I've added testing on GitHub Actions for linux-aarch64 and linux-riscv64. You'll note I had to downgrade JNA to 5.12 because of java-native-access/jna#1557.

BTW: well done for finding out how to do that!

@codecov-commenter
Copy link

codecov-commenter commented Oct 31, 2023

Codecov Report

Merging #264 (4c69119) into master (c2865b7) will not change coverage.
Report is 9 commits behind head on master.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #264   +/-   ##
=========================================
  Coverage     75.27%   75.27%           
  Complexity      441      441           
=========================================
  Files            38       38           
  Lines          1820     1820           
  Branches        178      178           
=========================================
  Hits           1370     1370           
  Misses          340      340           
  Partials        110      110           

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

Copy link
Contributor

@sebbASF sebbASF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments

src/docker/build.sh Outdated Show resolved Hide resolved
Copy link
Contributor

@sebbASF sebbASF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See review comment

pom.xml Show resolved Hide resolved
.github/workflows/maven.yml Outdated Show resolved Hide resolved
@sebbASF
Copy link
Contributor

sebbASF commented Nov 2, 2023

The cross-builds now seem to be working, though they do take a long time.
Are they doing more than they need?

Also it looks like they don't only build the native files for their own architectures.
They probably need their own build.sh files, or build.sh needs a parameter to control which native files are built

Copy link
Contributor

@sebbASF sebbASF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the docker run commands no longer need the -test services

src/docker/docker-compose.yaml Outdated Show resolved Hide resolved
src/docker/docker-compose.yaml Outdated Show resolved Hide resolved
@luhenry
Copy link
Contributor Author

luhenry commented Nov 2, 2023

The cross-builds now seem to be working, though they do take a long time.
Are they doing more than they need?

The step that takes the longest is setting up the java and openssl packages on the aarch64/riscv64 docker images. (ex: https://github.com/apache/commons-crypto/actions/runs/6733654561/job/18304390181#step:4:6004, 6 out of 10 minutes just on that step). This docker image could be cached by pushing it to the repo's docker repository. I've seen that done as part of https://github.com/uraimo/run-on-arch-action.

@sebbASF
Copy link
Contributor

sebbASF commented Nov 2, 2023

The cross-builds now seem to be working, though they do take a long time.
Are they doing more than they need?

The step that takes the longest is setting up the java and openssl packages on the aarch64/riscv64 docker images. (ex: https://github.com/apache/commons-crypto/actions/runs/6733654561/job/18304390181#step:4:6004, 6 out of 10 minutes just on that step). This docker image could be cached by pushing it to the repo's docker repository. I've seen that done as part of https://github.com/uraimo/run-on-arch-action.

Do we need the full JDK and the dev version of libssl?
Installing only the JVM and standard libssl might speed things up.

@luhenry
Copy link
Contributor Author

luhenry commented Nov 3, 2023

Do we need the full JDK and the dev version of libssl?

I've trimmed it down to openjdk-17-jre-headless coming from ubuntu packages and not a specific PPA, but I needed to keep libssl-dev as libssl1.1 only installs libcrypto.so.1.1 while the native commons-crypto library links against libcrypto.so, leading to a linker issue.

Overall, it goes from ~360s to ~110s locally. And on CI, it goes down to ~150s.

@sebbASF sebbASF merged commit d654a39 into apache:master Nov 4, 2023
18 checks passed
@sebbASF
Copy link
Contributor

sebbASF commented Nov 4, 2023

Thanks for your patience dedication getting this together!
Any further tweaks can be made in the main repo

@luhenry
Copy link
Contributor Author

luhenry commented Nov 4, 2023

@sebbASF Thank you for the review and continuous feedback as well! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants