FormEncoded Request fails (404 Not Found) if key value pairs order in mapping is different from request body order

[Xor-el]

Describe the bug

When trying to use the ExactMatcher or RegexMatcher to match a form encoded request, it fails (404 Not Found) if they order of the form encoded request is different from that in the json mapping file.

Expected behavior:

Ordering of FormEncoded Body requests should not affect matching as it might not always be convenient to control the ordering.
If it is going to be a breaking change to fix this, a flag like IgnoreFormEncodedBodyOrder can be provided to override the default.

Test to reproduce

• 1 Clone Repo, and build.
• 2 Run the built demo. (Incase you get an error the first time you run the Demo, close it and run again.).

Other related info

Docker is needed to run the Demo as it uses WireMock.NET Test Container.

[StefH]

The ExactMatcher or RegexMatcher should just be used to match a string value. In case you want to match a form-urlencoded body, a better approach would be to add a new matcher like FormUrlEncodedMatcher which can be used like this?

"Body": {
      "Matcher": {
        "Name": "FormUrlEncodedMatcher",
        "Patterns": [
           "grant_type=client_credentials",
           "client_id=DDCD99EE1531484E4E21D5EC9FBA5D8B",
           "client_secret=RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI%3D"
         ]
      }
    }

[Xor-el]

Hello @StefH thanks for your quick feedback.
I agree with your suggestion that adding a new matcher would be overall a better approach to address this.
Thanks for looking into this.

[StefH]

#1147

[StefH]

@Xor-el
Can you double check the PR (#1147)
And I've updated the wiki: https://github.com/WireMock-Net/WireMock.Net/wiki/Request-Matcher-FormUrlEncodedMatcher

[Xor-el]

Hello @StefH Reviewing the PR and Doc now.

[Xor-el]

So just reviewed this and everything looks awesome. Thanks.
while reviewing, a thought came to my head. does it make sense to have something like a FormUrlEncodedPartialMatcher, similar to how JsonPartialMatcher does partial matching or does the MatchOperator functionality cover this?

[StefH]

By default the MatchOperator is set to Or.
So when the matcher is like:

 "Patterns": [
           "grant_type=client_credentials",
           "client_id=DDCD99EE1531484E4E21D5EC9FBA5D8B",
           "client_secret=RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI%3D"
         ]

And the body only contains grant_type=client_credentials, this matcher will match.

---

A FormUrlEncodedPartialMatcher could be used like this (I think)...

"Patterns": [
           "grant_type=*",
           "client_id=DD*",
           "*=RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI%3D"
         ]

I'll check if I can change the implementation from the FormUrlEncodedMatcher to use internally the WildcardMatcher for string compare...

[Xor-el]

and if it's the other way,

"Patterns": [
           "grant_type=client_credentials"
         ]

And the body contains grant_type=client_credentials, client_id=DDCD99EE1531484E4E21D5EC9FBA5D8B, and client_secret=RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI%3D I am guessing the matching would still work right or only the FormUrlEncodedPartialMatcher would work then?

[StefH]

and if it's the other way,

"Patterns": [
           "grant_type=client_credentials"
         ]

And the body contains grant_type=client_credentials, client_id=DDCD99EE1531484E4E21D5EC9FBA5D8B, and client_secret=RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI%3D I am guessing the matching would still work right or only the FormUrlEncodedPartialMatcher would work then?

This does already work like it is because Or is used.

[Xor-el]

great then, thanks.

[StefH]

I've just updated the code so that wildcards are supported.
See the FormUrlEncodedMatcherTest for details.

[Xor-el]

LGTM 🚀

[Xor-el]

Hello @StefH I updated the test container package to version 1.5.62 which I assume contains the FormUrlEncodedMatcher changes.
unfortunately, the matching still returns 404 error.

I updated the Demo to reflect this issue.
can you please confirm if I am missing something?

[StefH]

I could not test using docker container, but I just started latest dotnet-wiremock and updated your program.cs to be like

// See https://aka.ms/new-console-template for more information
using Newtonsoft.Json;
// using WireMock.Net.Testcontainers;

//var container = new WireMockContainerBuilder().WithMappings(Helper.MappingsPath)
//                                              .WithAutoRemove(true)
//                                              .WithCleanUp(true)
//                                              .Build();

//await container.StartAsync().ConfigureAwait(false);

//var publicBaseUrl = container.GetPublicUrl() + "api";

//Console.WriteLine("baseUrl = " + publicBaseUrl);

//var wireMockAdminClient = container.CreateWireMockAdminClient();

//var settings = await wireMockAdminClient.GetSettingsAsync();
//Console.WriteLine("settings = " + JsonConvert.SerializeObject(settings, Formatting.Indented));

//var mappings = await wireMockAdminClient.GetMappingsAsync();
//Console.WriteLine("mappings = " + JsonConvert.SerializeObject(mappings, Formatting.Indented));

var client = new HttpClient();
var publicBaseUrl = "http://localhost:9091/api";
client.BaseAddress = new Uri(publicBaseUrl);

Console.WriteLine("=================================Starting OrderedContent Tests=================================");

// grant_type is first both here and in the mapping file
var orderedContent = new FormUrlEncodedContent(
[
    new KeyValuePair<string, string>("grant_type", "client_credentials"),
    new KeyValuePair<string, string>("client_id", "DDCD99EE1531484E4E21D5EC9FBA5D8B"),
    new KeyValuePair<string, string>("client_secret", "RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI=")
]);

//var exactOrderedContentResponse = await client.PostAsync($"{publicBaseUrl}/exact", orderedContent).ConfigureAwait(false);

//Console.WriteLine("ExactOrderedContent Response = " + exactOrderedContentResponse.StatusCode);

//var regexOrderedContentResponse = await client.PostAsync($"{publicBaseUrl}/regex", orderedContent).ConfigureAwait(false);

//Console.WriteLine("RegexOrderedContent Response = " + regexOrderedContentResponse.StatusCode);

var formUrlEncodedOrderedContentResponse = await client.PostAsync($"{publicBaseUrl}/formurlencoded", orderedContent).ConfigureAwait(false);

Console.WriteLine("FormUrlEncodedOrderedContent Response = " + formUrlEncodedOrderedContentResponse.StatusCode);

Console.WriteLine("=================================Starting UnOrderedContent Tests=================================");

// order changed, grant_type is now last here but still first in mapping file
var unOrderedContent = new FormUrlEncodedContent(
[
    new KeyValuePair<string, string>("client_id", "DDCD99EE1531484E4E21D5EC9FBA5D8B"),
    new KeyValuePair<string, string>("client_secret", "RERDRDk5RUUxNTMxNDg0RTRFMjFENUVDOUZCQTVEOEI="),
    new KeyValuePair<string, string>("grant_type", "client_credentials"),
]);

//var exactUnOrderedContentResponse = await client.PostAsync($"{publicBaseUrl}/exact", unOrderedContent).ConfigureAwait(false);

//Console.WriteLine("ExactOrderedContent Response = " + exactUnOrderedContentResponse.StatusCode);

//var regexUnOrderedContentResponse = await client.PostAsync($"{publicBaseUrl}/regex", unOrderedContent).ConfigureAwait(false);

//Console.WriteLine("RegexOrderedContent Response = " + regexUnOrderedContentResponse.StatusCode);

var formUrlEncodedUnOrderedContentResponse = await client.PostAsync($"{publicBaseUrl}/formurlencoded", unOrderedContent).ConfigureAwait(false);

Console.WriteLine("FormUrlEncodedUnOrderedContent Response = " + formUrlEncodedUnOrderedContentResponse.StatusCode);


//await container.StopAsync();

Console.WriteLine("Finished Operations");

Console.ReadLine();

public static class Helper
{
    private static Lazy<string> RootProjectPath { get; } = new(() =>
    {
        var directoryInfo = new DirectoryInfo(AppContext.BaseDirectory);
        do
        {
            directoryInfo = directoryInfo.Parent!;
        } while (!directoryInfo.Name.EndsWith("WireMock.NET-FormEncoded-Body-Bug-Demo", StringComparison.OrdinalIgnoreCase));

        return directoryInfo.FullName;
    });

    public static string MappingsPath => Path.Combine(RootProjectPath.Value, "Mappings");
}

And it works:

---

In case you want to know why the mapping is not matched, follow this page:
https://github.com/WireMock-Net/WireMock.Net/wiki/Request-Matching-Tips

[Xor-el]

@StefH my bad actually. I forgot to clear my Stale Docker Image Cache so my test container was using the old cached 1.5.60 instead of the latest 1.5.62 image.

Thanks for your time.

[StefH]

@Xor-el
Currently the latest is always used.
Would it be needed to add an option to manually specify the version when using the containerbuilder? Or is this not needed?

[Xor-el]

@StefH

Currently the latest is always used.

So, Docker Desktop normally caches the downloaded image and reuses it on subsequent builds if it exists so I don't think there is much you can do about that on your end except there is a way to force Docker to always pull when a build is about to happen (but this has it's own disadvantage too).

Would it be needed to add an option to manually specify the version when using the containerbuilder? Or is this not needed?

it would be nice to have the ability to pin the version we wish to use to enable users to work around scenarios like this, which happened some days back to the SQL Server test container image.

[StefH]

Can you create a new issue please?

[Xor-el]

sure, on it.