Linux cuda crash

[ChengYen-Tang]

I'm getting a crash using LLamaSharp.Backend.Cuda11
This problem occurs in 0.6.0 and 0.7.0, and is normal in 0.5.1

Env:
Ubuntu 22.04

[martindevans]

We have a new set of binaries ready to release in the next version (very soon). Could you try pulling the master branch and testing if it fixes this issue?

[ChengYen-Tang]

Same error

[martindevans]

What CPU and GPU are you using?

Edit: Original post shows Titan RTX for the GPUs

[ChengYen-Tang]

CPU: I7-8700
GPU: Titan RTX 24G

[AsakusaRinne]

It's quite weird because the only difference on compilation of native library since v0.6.0 is avx.

We've published v0.8.0 just now, with a fix for cuda library (but not sure if it's helpful for this issue) and the cuda feature detection in #275. Could you please have a try with it?

If it still doesn't work, you could add NativeLibraryConfig.WithLibrary({Your_DLL_PATH}) at the very beginning of your code, to load a cuda library compiled yourself. That could be a temporary fix.

[ChengYen-Tang]

Oh, it’s normal
Another question, how should I stop printing logs?

[AsakusaRinne]

Oh, it’s normal

Is the v0.8.0 normal or your self-compiled library normal?

Another question, how should I stop printing logs?

Stopping logs from llama.cpp is an another issue. Currently you could try to redirect the outputs to other where.

[AsakusaRinne]

Update: this comment will allow you to capture the outputs from native library and output them anywhere else.

[ChengYen-Tang]

Oh, it’s normal

Is the v0.8.0 normal or your self-compiled library normal?

Another question, how should I stop printing logs?

Stopping logs from llama.cpp is an another issue. Currently you could try to redirect the outputs to other where.

v0.8.0

Update: this comment will allow you to capture the outputs from native library and output them anywhere else.

Thank you.