Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Microsoft.Extensions.AI] Add EndUserId as tag to span/traces/metrics #5583

Open
kzu opened this issue Oct 29, 2024 · 1 comment
Open

[Microsoft.Extensions.AI] Add EndUserId as tag to span/traces/metrics #5583

kzu opened this issue Oct 29, 2024 · 1 comment

Comments

@kzu
Copy link
Contributor

kzu commented Oct 29, 2024

It's typically quite useful to be able to associate genai telemetry with a particular user, for multiple reasons (i.e. track token consumption per user, rate limit per user, etc.). OpenAI and Claude support such an identifier specifically, but I couldn't find a similar mechanism for Gemini or Cohere.

While this is easy to do add this tag for the activity/span itself (via a delegating chat client that adds that to the Activity.Current), it's impossible to do the same for the token consumption and operation duration metrics.

It would be ideal if arbitrary (serializable/primitive?) properties in the ChatOptions.AdditionalProperties were automatically added to both traces and metrics automatically by the OpenTelemetryChatClient, or alternatively, at least just the EndUserId. It could also be an explicit setting for the client, to be configured just like EnableSensitiveData, such as EnableAdditionalProperties or EnableAdditionalTags).


Workaround: how to add EndUserId to traces (not metrics though)

    class UserIdChatClient(IChatClient client) : DelegatingChatClient(client)
    {
        public override Task<Microsoft.Extensions.AI.ChatCompletion> CompleteAsync(IList<Microsoft.Extensions.AI.ChatMessage> chatMessages, ChatOptions? options = null, CancellationToken cancellationToken = default)
        {
            if (Activity.Current is { } activity && options?.AdditionalProperties?.TryGetValue("EndUserId", out var endUserId) == true)
                activity.SetTag("user.id", endUserId);

            return base.CompleteAsync(chatMessages, options, cancellationToken);
        }

        public override IAsyncEnumerable<Microsoft.Extensions.AI.StreamingChatCompletionUpdate> CompleteStreamingAsync(IList<Microsoft.Extensions.AI.ChatMessage> chatMessages, ChatOptions? options = null, CancellationToken cancellationToken = default)
        {
            if (Activity.Current is { } activity && options?.AdditionalProperties?.TryGetValue("EndUserId", out var endUserId) == true)
                activity.SetTag("user.id", endUserId);

            return base.CompleteStreamingAsync(chatMessages, options, cancellationToken);
        }
    }

And ensure it's added right after the open telemetry client:

services.AddChatClient(builder => builder
        .UseOpenTelemetry()
        .Use(client => new UserIdChatClient(client))
        .Use(new OpenAIClient(...));
@kzu kzu added the untriaged label Oct 29, 2024
@stephentoub
Copy link
Member

@lmolkova / @samsp-msft, what would you recommend here? From the perspective of adhering to the genai semantic conventions, is it ok / recommended to add additional tags?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants