Autorenew OAuth token

[maxkoshevoi]

If I provide Flurl with Access Token URL, Client ID, and Client Secret, can it

• obtain the OAuth token and cache it
• send it with the request
• use cached token when new request it issued with the same Access Token URL, Client ID, and Client Secret
• obtain new token when old one expires

[bgmulinari]

This would be really nice to have. Right now you can do this with IdentityModel.AspNetCore, but it's a bit awkward making it work together with Flurl as Flurl doesn't support named httpClient as far as I'm aware.

[maxkoshevoi]

My manual implementation looks like this:

Usage:

public async Task<List<IdentityUserInfo>> GetUsersAsync(IEnumerable<Guid> userIds)
{
    var request = await GetBaseRequestAsync();
    var response = await request
        .AppendPathSegment("user/getUsers")
        .PostJsonAsync(new { UserIds = userIds.ToArray() })
        .ReceiveJson<GetUsersResponse>();
    return response.Users;
}

Behind the scenes:

    private static readonly SemaphoreSlim _accessTokenSemaphore = new(1, 1);
    private static AccessTokenModel? _accessToken;

    protected async Task<IFlurlRequest> GetBaseRequestAsync()
    {
        return _settings.IdentityProviderUrl
            .WithOAuthBearerToken(await GetAccessTokenAsync(_settings.OAuth))
            .OnError(ThrowInvalidOperationAsync);

        static async Task ThrowInvalidOperationAsync(FlurlCall call)
        {
            var errorMessage = await call.Response.GetStringAsync();
            throw new InvalidOperationException(errorMessage);
        }
    }

    private static async Task<string> GetAccessTokenAsync(OAuthOptions oAuth)
    {
        if (_accessToken is not { Expired: false })
        {
            _accessToken = await FetchTokenAsync();
        }

        return _accessToken.AccessToken;

        async Task<AccessTokenModel> FetchTokenAsync()
        {
            try
            {
                await _accessTokenSemaphore.WaitAsync();

                return await oAuth.ServerUrl
                    .AppendPathSegment("OAuth/Token")
                    .PostUrlEncodedAsync(new
                    {
                        client_id = oAuth.ClientId,
                        client_secret = oAuth.ClientSecret,
                        grant_type = "client_credentials"
                    })
                    .ReceiveJson<AccessTokenModel>();
            }
            finally
            {
                _accessTokenSemaphore.Release(1);
            }
        }
    }

    private record AccessTokenModel(string AccessToken, string TokenType, DateTime Expires)
    {
        // Let token "expire" 1 minute before it's actual expiration
        // to avoid using expired tokens and getting 401.
        private static readonly TimeSpan _threshold = TimeSpan.FromMinutes(1);

        [JsonConstructor]
        public AccessTokenModel(string access_token, string token_type, int expires_in)
            : this(access_token, token_type, DateTime.UtcNow.AddSeconds(expires_in))
        {
        }

        public bool Expired => Expires - DateTime.UtcNow <= _threshold;
    }

[bgmulinari]

Interesting. In my case I wanted to avoid having to manage the token myself, so for now this is what I came up with:

It's a .NET 6 API, so pretty much everything is configured in my Program.cs

using Flurl.Http;
using Flurl.Http.Configuration;
using IdentityModel.Client;
using IHttpClientFactory = System.Net.Http.IHttpClientFactory;

const string apiClientName = "MyApiClient";
var builder = WebApplication.CreateBuilder(args);

...

// Auto token management
builder.Services.AddAccessTokenManagement(options =>
{
    options.Client.Clients.Add(apiClientName, new ClientCredentialsTokenRequest
    {
        Address = "http://myapi.com/oauth/token",
        ClientId = "my_client_id",
        ClientSecret = "my_client_secret",
        Scope = "my_scope"
    });
});

// Adds a named HTTP client for the factory that automatically sends the client access token
builder.Services.AddClientAccessTokenHttpClient(
    apiClientName, apiClientName, client => client.BaseAddress = new Uri("http://myapi.com"));

...

var app = builder.Build();
var scope = app.Services.CreateScope();

// Custom FlurlClient configuration to set my custom HttpClientFactory
FlurlHttp.ConfigureClient("http://myapi.com", client =>
{
    var httpClientFactory = scope.ServiceProvider.GetService<IHttpClientFactory>();
    client.Settings.HttpClientFactory = new CustomHttpClientFactory(apiClientName, httpClientFactory);
});

And here's the CustomHttpClientFactory class:

internal class CustomHttpClientFactory : DefaultHttpClientFactory
{
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly string _clientName;

    public CustomHttpClientFactory(string clientName, IHttpClientFactory httpClientFactory)
    {
        _clientName = clientName;
        _httpClientFactory = httpClientFactory;
    }

    public override HttpClient CreateHttpClient(HttpMessageHandler handler)
    {
        return _httpClientFactory.CreateClient(_clientName);
    }
}

With this, every request made with Flurl to "http://myapi.com" will have the token managed automatically. However, one problem I see is that in my override CustomHttpClientFactory.CreateHttpClient I don't call base.CreateClient, which as noted in Flurl comments is not recommended as I might lose Flurl.Http functionality.

So not an ideal implementation, but it works for me. I'd be interested to know @tmenier's take on this.

[tmenier]

I'll look into this for 4.x. I can't claim that Flurl is OAuth-agnostic since it already has WithOAuthBearerToken, so taking a step deeper into that water doesn't seem totally unreasonable. :) Also, you could say a goal of mine is for Flurl to be a great option for building SDKs for APIs and this would further that goal.

[rcollette]

Prior comment on the topic of token renewal.
#619 (comment)

[alekdavis]

I have the same requirement (auto-refresh bearer tokens) and after looking at available options, I am leaning towards implementing this myself (I first tried MSAL.NET and it seemed to work fine, but unfortunately, it only works with Azure tokens and I need something that supports any custom token endpoint). So here is my idea.

In the nutshell, when I get a bearer token, the expires_in property returns the number of seconds the token will be valid. Since I know when the request for the token was made, I can calculate when it is set to expire. It may not be precise, but since bearer tokens generally have a lifespan of about an hour or so, a minute or two make little difference.

For each authentication endpoint, I would have a service responsible for creating and refreshing bearer tokens. The authentication service would use two helper classes: one for authentication endpoint settings (URL, client ID, client secret, etc), another for access token properties (token, expiration date, etc). So whenever I make a call to some endpoint, I pass a helper method that either retrieves an existing bearer token from the authentication service or have the authentication service return a refreshed one (by default, it would refresh the token 5 minutes before the calculated expiration).

I'm still working on it (don't mind Console.WriteLine), but here is the code prototype.

On the client side, the calls would be like:

 var dataOut= await someEndpointUrl
    .WithOAuthBearerToken(someEndpointTokenService?.GetValidToken()?.Token ?? "")
    .PostJsonAsync(dataIn)
    .ReceiveJson<DataOut>();

The endpoint token service instance would be defined before that call as:

AccessTokenServiceSettings someEndpointTokenServiceSettings = new AccessTokenServiceSettings(
        "https://xxx.com/v1/auth/token", // auth token URL
        "XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXX",  // client ID
        "XXXXXXXXX", // client secret
        new string[] { ".default" } ); // scopes

// The constructor would generate the first token.
AccessTokenService someEndpointTokenService = new AccessTokenService(someEndpointTokenServiceSettings);

And here are the three classes I mentioned above (do not mind Console calls, since it's just a prototype in a console app):

public class AccessToken
{
    private DateTime _validFrom;
    private DateTime _validTo;
    private string _token;
    private int _secondsToLive;

    public DateTime ValidFrom { get { return _validFrom; } }

    public DateTime ValidTo { get { return _validTo; } }
    
    public string Token { get { return _token; } }

    public int SecondsToLive { get { return _secondsToLive; } }

    public AccessToken
    (
        OAuthResponse   oAuthResponse,
        DateTime?       validFrom = null
    )
    {
        if (oAuthResponse == null)
            throw new ArgumentNullException(nameof(oAuthResponse));

        if (String.IsNullOrEmpty(oAuthResponse.expires_in))
            throw new ArgumentNullException(nameof(oAuthResponse.expires_in));

        if (String.IsNullOrEmpty(oAuthResponse.access_token))
            throw new ArgumentNullException(nameof(oAuthResponse.access_token));

        if (validFrom == null)
        {
            _validFrom = DateTime.UtcNow;
        }
        else
        {
            _validFrom = validFrom.Value;
        }

        double expiresIn;
        
        try
        {
            expiresIn = double.Parse(oAuthResponse.expires_in);
            _secondsToLive = int.Parse(oAuthResponse.expires_in);
        }
        catch (Exception ex)
        {
            throw new Exception(
                $"Cannot convert the string value of '{nameof(oAuthResponse.expires_in)}' holding '{oAuthResponse.expires_in}' to a number.", ex);
        }

        _validTo = _validFrom.AddSeconds(expiresIn);

        _token = oAuthResponse.access_token;
    }
}

public class AccessTokenServiceSettings
{
    public string? Url          = null;
    public string? ClientId     = null;
    public string? ClientSecret = null;
    public string[]? Scopes     = null;
    public IWebProxy? Proxy     = null;

    public AccessTokenServiceSettings
    (
        string? url,
        string? clientId,
        string? clientSecret,
        string[]? scopes = null,
        WebProxy? proxy = null
    )
    {
        Url = url;
        ClientId = clientId;
        ClientSecret = clientSecret;
        Scopes = scopes;
        Proxy = proxy;
    }
}

public class AccessTokenService
{
    protected string? _url              = null;
    protected string? _clientId         = null;
    protected string? _clientSecret     = null;
    protected string[]? _scopes         = null;
    protected IWebProxy? _proxy         = null;
    protected AccessToken? _accessToken = null;
    protected HttpClient? _httpClient   = null;

    public AccessTokenService
    (
        AccessTokenServiceSettings settings
    )
    :
    this(settings.Url, settings.ClientId, settings.ClientSecret, settings.Scopes, settings.Proxy)
    {
    }

    public AccessTokenService
    (
        string? url,
        string? clientId,
        string? clientSecret,
        string[]? scopes = null,
        IWebProxy? proxy = null
    )
    {
        _url        = url;
        _clientId   = clientId;
        _clientSecret= clientSecret;
        _scopes     = scopes;
        _proxy      = proxy;

        Refresh();
    }

    public AccessToken? AccessToken
    {
        get
        {
            return _accessToken;
        }
    }

    public AccessToken? GetValidToken
    (
        int secondsBeforeExpiration = 300,
        int secondsAfterCreation = 0
    )
    {
        AccessToken? accessToken = null;

        if (_accessToken != null && IsValidToken())
        {
            accessToken = _accessToken;
        }

        if (_accessToken == null || !IsValidToken())
        {
            Console.WriteLine("Access token is null or invalid.");

            Refresh();

            accessToken = _accessToken;
        }
        else if (MustRenewToken(secondsBeforeExpiration, secondsAfterCreation))
        {
            try
            {
                Console.WriteLine("Access token must be renewed.");

                Refresh();

                accessToken = _accessToken;
            }
            catch (Exception ex)
            {
                Console.WriteLine("Cannot renew access token.");

                // TODO: Non-fatal error
                Console.WriteLine(ex.GetMessages());
            }
        }

        return accessToken;
    }

    public void Refresh()
    {
        if (_httpClient == null)
        {
            HttpClientHandler httpHandler = new HttpClientHandler() { UseDefaultCredentials = false };

            if (_proxy != null)
            {
                httpHandler.Proxy       = _proxy;
                httpHandler.UseProxy    = true;
            }

            _httpClient = new HttpClient(httpHandler);
        }

        _httpClient.DefaultRequestHeaders.Clear();
        _httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/x-www-form-urlencoded"));
        _httpClient.DefaultRequestHeaders.Add(
            "Authorization",
            "Basic " + Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes($"{_clientId}:{_clientSecret}")));

        List<KeyValuePair<string, string>> data = new List<KeyValuePair<string, string>>()
        {
            new KeyValuePair<string,string>("grant_type", "client_credentials"),
        };

        if (_scopes != null && _scopes.Length > 0)
        {
            data.Add(new KeyValuePair<string, string>("scope", String.Join(" ", _scopes)));
        }

        FormUrlEncodedContent content = new FormUrlEncodedContent(data);

        try
        {
            DateTime timestamp = DateTime.UtcNow;

            Task tasks;

            Task<HttpResponseMessage> postTask = _httpClient.PostAsync(_url, content);

            tasks = Task.WhenAll(postTask);
            
            try
            {
                if (_scopes == null || _scopes.Length == 0)
                    Console.WriteLine($"Posting client credentials for '{_clientId}' to token endpoint '{_url}'.");
                else
                    Console.WriteLine($"Posting client credentials for '{_clientId}' with scopes '{String.Join(" ", _scopes)}' to token endpoint '{_url}'.");

                tasks.Wait();
            }
            catch (Exception ex)
            {
                throw new Exception("Cannot post client credentials to the token endpoint.", 
                    (ex is AggregateException) ? ex.InnerException : ex);
            }

            HttpResponseMessage response = postTask.Result;

            Task<String> stringTask = response.Content.ReadAsStringAsync();

            tasks = Task.WhenAll(stringTask);

            try
            {
                tasks.Wait();
            }
            catch (Exception ex)
            {
                throw new Exception("Cannot get response content for from the token endpoint.", 
                    (ex is AggregateException) ? ex.InnerException : ex);
            }

            string json = stringTask.Result;

            if (response.IsSuccessStatusCode)
            {
                OAuthResponse? oauthResponse = JsonConvert.DeserializeObject<OAuthResponse>(json);

                if (oauthResponse == null)
                {
                    throw new Exception($"Cannot deserialize access token from the response content: {json}");
                }
                else
                {
                    try
                    {
                        _accessToken = new AccessToken(oauthResponse, timestamp);
                    }
                    catch (Exception ex)
                    {
                        throw new Exception($"Cannot initialize access token from the response content: {json}.", ex);
                    }
                }
            }
            else
            {
                OAuthError oauthError = new OAuthError(json);

                if (oauthError == null)
                {
                    throw new Exception($"The POST operation failed, but error object could not be deserialized from the response content: {json}");
                }
                else
                {
                    if (_scopes == null || _scopes.Length == 0)
                        throw new OAuthException(oauthError);
                    else
                        throw new OAuthException(oauthError);
                }
            } 
        }
        catch (Exception ex)
        {
            if (_scopes == null || _scopes.Length == 0)
                throw new Exception($"Cannot get access token for '{_clientId}' from '{_url}'.", ex);
            else
                throw new Exception($"Cannot get access token for '{_clientId}' with scopes '{String.Join(" ", _scopes)}' from '{_url}'.", ex);
        }
    }

    public bool IsValidToken()
    {
        if (_accessToken == null)
            return false;

        DateTime now = DateTime.UtcNow;

        return (_accessToken.ValidFrom <= now && now <= _accessToken.ValidTo);
    }

    public bool MustRenewToken
    (
        int secondsBeforeExpiration = 300,
        int secondsAfterCreation = 0
    )
    {
        if (_accessToken == null)
            return true;

        DateTime now = DateTime.UtcNow;
        DateTime validTo;

        if (secondsBeforeExpiration <= 0)
            secondsBeforeExpiration = 0;

        if (secondsAfterCreation > 0)
        {
            if (_accessToken.SecondsToLive - secondsBeforeExpiration > secondsAfterCreation)
            {
                validTo = _accessToken.ValidFrom.AddSeconds(secondsAfterCreation);
            }
            else
            {
                validTo = _accessToken.ValidFrom.AddSeconds(_accessToken.SecondsToLive - secondsBeforeExpiration);
            }
        }
        else
        {
            validTo = _accessToken.ValidFrom.AddSeconds(_accessToken.SecondsToLive - secondsBeforeExpiration);
        }

        return !(_accessToken.ValidFrom <= now && now <= validTo);
    }

    public bool IsTokenExpired
    {
        get
        {
            if (_accessToken == null)
                return false;

            DateTime now = DateTime.UtcNow;

            return (now > _accessToken.ValidTo);
        }
    }
}

Oh, and the helper classes for handling OAuth calls;

public class OAuthError
{
    public OAuthError
    (
        string json
    )
    {
        Raw? raw = JsonConvert.DeserializeObject<Raw>(json);

        if (raw != null)
        {
            Error = raw.error ?? raw.errorMessage;

            Description = raw.error_description ?? raw.errorReason;

            Code = raw.code ?? raw.errorStatus;

            if (Code == null && raw.error_codes != null && raw.error_codes.Length > 0)
            {
                Code = raw.error_codes[0];
            }
        }
    }

    public string? Error { get; set; }

    public string? Description { get; set; }

    public int? Code { get; set; }

    private class Raw
    {
        // Azure OAUTH
	public string? error { get; set; }

        public string? error_description { get; set; }

        public int? code { get; set; }

        public int[]? error_codes { get; set; }
    
        public DateTime? timestamp { get; set; }
    
        public string? trace_id { get; set; }
    
        public string correlation_id { get; set; }
   
        public string error_uri { get; set; }

        // Apigee OAUTH
        public string? errorMessage { get; set; }

        public string? errorReason { get; set; }

        public int? errorStatus { get; set; }
    }
}

public class OAuthException: Exception
{
    private static string _defaultMessage = "Authentication error occurred.";

    public OAuthError? Error { get; set; }

    public OAuthException
    (
        string message,
        OAuthError error
    )
    :
    base(message)
    {
        Error = error;
    }

    public OAuthException
    (
        OAuthError error
    )
    :
    base(FormatMessage(error))
    {
        Error = error;
    }

    public OAuthException
    (
        OAuthError error,
        Exception ex
    )
    :
    base(FormatMessage(error), ex)
    {
        Error = error;
    }

    private static string FormatMessage
    (
        OAuthError error
    )
    {
        if (error == null)
            return _defaultMessage;

        if (String.IsNullOrEmpty(error.Error) &&
            String.IsNullOrEmpty(error.Description) &&
            !error.Code.HasValue)
            return _defaultMessage;

        if (String.IsNullOrEmpty(error.Error) &&
            String.IsNullOrEmpty(error.Description) &&
            error.Code.HasValue)
        {
            return $"Error code '{error.Code.Value}' returned.";
        }

        if (!String.IsNullOrEmpty(error.Error) &&
            !String.IsNullOrEmpty(error.Description))
        {
            string msg = $"Error '{error.Error}' returned: {error.Description}".TrimEnd();

            if (Regex.IsMatch(msg, "[a-zA-Z0-9]$"))
                msg += ".";

            return msg;
        }

        if (!String.IsNullOrEmpty(error.Error))
        {
            return $"Error '{error.Error}' returned.";
        }

        return error.Description ?? _defaultMessage;
    }
}

public class OAuthResponse
{
    public string token_type { get; set; }

    public string expires_in { get; set; }

    public string ext_expires_in { get; set; }

    public string expires_on { get; set; }

    public string not_before { get; set; }

    public string resource { get; set; }

    public string access_token { get; set; }
}

public static class Extension
{
    public static string? ToJsonString
    (
        this object data,
        bool formatted =  false
    )
    {
        if (data == null)
            return null;

        JsonSerializer json = new JsonSerializer()
        {
            NullValueHandling = NullValueHandling.Ignore,
            DateFormatString = "yyyy-MM-ddTHH:mm:ss.fffZ",
            Formatting = formatted ? Formatting.Indented : Formatting.None,
            ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
            PreserveReferencesHandling = PreserveReferencesHandling.Objects
        };

        var textWriter = new StringWriter();

        json.Serialize(textWriter, data);

        return textWriter.ToString();
    }

    public static T? Clone<T>(this T source)
    {
        var serialized = JsonConvert.SerializeObject(source);

        if (serialized == null)
            return default(T);

        return JsonConvert.DeserializeObject<T>(serialized);
    }

    public static string? GetMessages
    (
        this Exception ex, 
        bool fromInnerExceptionsOnly = false
    )
    {
        Exception e = ex;

        if (e == null)
            return "";

        // Skip aggregate exception message (it's not meaningful).
        if (e is AggregateException)
        {
#pragma warning disable CS8600 // Converting null literal or possible null value to non-nullable type.
            e = e.InnerException;
#pragma warning restore CS8600 // Converting null literal or possible null value to non-nullable type.
        }

        StringBuilder messages = new StringBuilder();

        if (e != null && !fromInnerExceptionsOnly)
        {
            messages.Append(e.Message);
#pragma warning disable CS8600 // Converting null literal or possible null value to non-nullable type.
            e = e.InnerException;
#pragma warning restore CS8600 // Converting null literal or possible null value to non-nullable type.
        }

        while (e != null)
        {
            // Do not append duplicate message.
            if (!messages.ToString().EndsWith(e.Message))
            {
                if (messages.Length > 0)
                {
                    messages.Append(" ");
                }
                messages.Append(e.Message);
            }

#pragma warning disable CS8600 // Converting null literal or possible null value to non-nullable type.
            e = e.InnerException;
#pragma warning restore CS8600 // Converting null literal or possible null value to non-nullable type.
        }

        return messages.ToString();
    }
}

[michalberezowski]

I'll look into this for 4.x. I can't claim that Flurl is OAuth-agnostic since it already has WithOAuthBearerToken, so taking a step deeper into that water doesn't seem totally unreasonable. :) Also, you could say a goal of mine is for Flurl to be a great option for building SDKs for APIs and this would further that goal.

It would be of lesser importance if Http named clients were supported - it should allow for use of existing libraries that do token lifecycle management, such as Duende.AccessTokenManagement? As far as I understand that is also in the works for 4.0?

[tmenier]

@michalberezowski There are plans in the works for an ASP.NET Core companion lib that would bring FlurlClient configuration all the way up to the service registration layer and follow many of the same patterns as IHttpClientFactory. But there's nothing stopping you from using named HttpClients today. Simply configure them at startup as you normally would, and design your service classes in a way that allows Flurl to take over from the constructor:

private readonly IFlurlClient _flurlClient;

public MyService(HttpClient httpClient)
{
    _flurlClient = new FlurlClient(httpClient);
}

[michalberezowski]

Thanks for replying, appreciate!

Regarding:

(...) But there's nothing stopping you from using named HttpClients today. Simply configure them at startup as you normally would, and design your service classes in a way that allows Flurl to take over from the constructor:

private readonly IFlurlClient _flurlClient;

public MyService(HttpClient httpClient)
{
    _flurlClient = new FlurlClient(httpClient);
}

...that is what I was trying to do, expanding on your comment on stackoverflow, but I think I misunderstood how the whole factoring of HttpClients works in ASP.NET Core DI, as I was getting something that looked like a default client injected and thus obviously it did not work for me. Also, I seem to remember finding another comment of yours, along the lines of "it currently doesn't work w/ named clients, planned for 4.0", so I've just put a pin in it, for the time being.

But I since realized I could instead inject the factory and create client/FlurlClient inside the service, and see if that would work. I will have to test it someday.

Sorry for slightly derailing the topic, wasn't the intention to turn it into troubleshooting my issue, let's end it here!

Thanks again for your time taken responding (and indeed, the time & skills needed to deliver us flUrl, in the first place! Very much appreciated!)

[tmenier]

Oops, you're right...the example I gave shows how you would get it to work with typed clients; it's slightly different with named clients but same basic idea - follow the pattern and wrap the HttpClient with Flurl as soon as you have it.

I seem to remember finding another comment of yours, along the lines of "it currently doesn't work w/ named clients"

I don't recall saying such a thing unless it was before the necessary constructor was introduced, but it's pretty straightforward armed with that.

[tmenier]

I'm cutting some of the bigger non-breaking enhancements originally planned for 4.0 in the interest of getting it released sooner. I still think this is a good idea and it'll be a strong contender to make the 4.1 cut though.