Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging interceptor #649

Merged
merged 8 commits into from
Apr 5, 2024
Merged

Conversation

martindevans
Copy link
Member

Rewritten logging callbacks so it can be configured without touching native methods. This allows it to be used before llama.cpp dll is loaded.

Resolves #645

@AsakusaRinne
Copy link
Collaborator

Will this work also suppress native logs from llava?

@martindevans
Copy link
Member Author

Unsure. It's just using the llama.cpp logging callback (llama_log_set), so if llava messages are not captured by it I think that would be a bug that would be easy to fix upstream.

LLama/Native/NativeApi.Load.cs Outdated Show resolved Hide resolved
…_empty_call`, this is specifically selected to be a method that does nothing and is just there for triggering DLL loading.
… always call whatever log callbacks you have registered.

 - Removed alternative path for `ILogger` in NativeLibraryConfig, instead it redirects to wrapping it in a delegate.
 - Using `ThreadLocal<StringBuilder>` to ensure messages from separate threads don't get mixed together.
@dlyz
Copy link
Contributor

dlyz commented Apr 5, 2024

Thanks @martindevans for your work! I have no further notes.

@martindevans
Copy link
Member Author

Thanks very much for your detailed review!

@martindevans martindevans merged commit 58107bb into SciSharp:master Apr 5, 2024
3 checks passed
@martindevans martindevans deleted the logging_interceptor branch April 5, 2024 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Can not intercept all the log messages with llama_log_set
3 participants