AWS .NET SDK response (VS) HttpClient response

[bagajjal]

I'm seeing discrepancies between the EC2 instance response from AWS .NET SDK (vs) using HTTPclient .
HTTPClient response is documented https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html

I'm under the impression that HTTPClient calls the REST API which will have the same schema as AWS .NET SDK but looks like it's not.
I would like to know where the schema is defined for the HTTPClient response.

Here is the comparison between .NET SDK and the HTTPClient. please note the response is sorted alphabetically.

[bagajjal]

@ashovlin - Any inputs?

[ashovlin]

Can you clarify where the JSON on the left side is from? Is that the SDK's DescribeInstancesResponse, serialized as JSON?

[bagajjal]

Yes. DescribeInstancesResponse serialized as JSON.

[bagajjal]

@ashishdhingra, can you please take a look at this.

[ashovlin]

We don't guarantee that the XML returned by a service, unmarshalled to the corresponding Response class in .NET, and then serialized again as JSON will be identical. I imagine there are even differences for services that return JSON instead of XML.

Common cases where they differ (as seen in the above example):

• Enumerations - What a service may think of as an enum, we actually use a class based on our ConstantClass. This offers more flexibility to clients who are using an older SDK that is not aware of a new enum value that a service may start returning.
  • Here's where the ArchitectureType enumeration is currently defined:

    aws-sdk-net/sdk/src/Services/EC2/Generated/ServiceEnumerations.cs

    Lines 906 to 962 in f02c1b2

    	public class ArchitectureType : ConstantClass
    	{
    	/// <summary>
    	/// Constant Arm64 for ArchitectureType
    	/// </summary>
    	public static readonly ArchitectureType Arm64 = new ArchitectureType("arm64");
    	/// <summary>
    	/// Constant Arm64_mac for ArchitectureType
    	/// </summary>
    	public static readonly ArchitectureType Arm64_mac = new ArchitectureType("arm64_mac");
    	/// <summary>
    	/// Constant I386 for ArchitectureType
    	/// </summary>
    	public static readonly ArchitectureType I386 = new ArchitectureType("i386");
    	/// <summary>
    	/// Constant X86_64 for ArchitectureType
    	/// </summary>
    	public static readonly ArchitectureType X86_64 = new ArchitectureType("x86_64");
    	/// <summary>
    	/// Constant X86_64_mac for ArchitectureType
    	/// </summary>
    	public static readonly ArchitectureType X86_64_mac = new ArchitectureType("x86_64_mac");
    	/// <summary>
    	/// This constant constructor does not need to be called if the constant
    	/// you are attempting to use is already defined as a static instance of
    	/// this class.
    	/// This constructor should be used to construct constants that are not
    	/// defined as statics, for instance if attempting to use a feature that is
    	/// newer than the current version of the SDK.
    	/// </summary>
    	public ArchitectureType(string value)
    	: base(value)
    	{
    	}
    	/// <summary>
    	/// Finds the constant for the unique value.
    	/// </summary>
    	/// <param name="value">The unique value for the constant</param>
    	/// <returns>The constant for the unique value</returns>
    	public static ArchitectureType FindValue(string value)
    	{
    	return FindValue<ArchitectureType>(value);
    	}
    	/// <summary>
    	/// Utility method to convert strings to the constant class.
    	/// </summary>
    	/// <param name="value">The string value to convert to the constant class.</param>
    	/// <returns></returns>
    	public static implicit operator ArchitectureType(string value)
    	{
    	return FindValue(value);
    	}
    	}

• Collections - EC2 denotes items in the collection via <item></item> Once we convert that to a List<T>, and then serialize that to JSON,, it'll appear as a JSON array [ {...}, {...}, {...} ]

The best documentation for the raw responses are each service's API reference: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html.

I'm curious about your use case, if you're able to share?

[bagajjal]

@normj, @ashishdhingra , I'm interested in only reading the AWS resources belonging to different AWS services.

If we opt for the .NET SDK approach, it entails creating code to read individual AWS resource types. This process would be ongoing as we continue to expand our scope of interested AWS resources.

I explored the HTTP client to make REST API calls, which initially seemed promising. However, I realized that the response lacks a predefined schema. Unlike the AWS .NET SDK, which employs service models to outline expected properties for a given resource, this schema is absent for the raw HTTP response.

A potential approach I'm considering involves checking whether the .NET SDK offers a generic API (not tied to any specific resource type) capable of accepting an HTTP response and returning a .NET object. This would enable utilizing .NET reflection to interpret the .NET object, as the properties are specific to the resource type.

Let me know if it makes sense or if there are better ways to solve this problem.

Thanks for your time.

[normj]

@bagajjal You might want to take a look at the AWS Cloud Control API also available in the SDK via the AWSSDK.CloudControlApi NuGet package. This service provides a generic API for doing CRUD operations for AWS resources. It doesn't cover every AWS service but a pretty high percentage of them.

[bagajjal]

@normj , I looked into the AWS Cloud control APIs. The problem with AWS cloud control APIs is similar to HTTP client raw response, the schema of the response i.e., ResourceDescriptions is not documented per resource type.

[bagajjal]

@normj , @ashishdhingra , Could you please refer me to the unmarshaling APIs. Are these APIs specific to resource types or a generic API?

[normj]

@bagajjal In every service package their is a Amazon..Model.Internal.MarshallTransforms namespace that contains the generated code for marshalling the the .NET types and the wire format. Here is EC2's marshallers https://github.com/aws/aws-sdk-net/tree/master/sdk/src/Services/EC2/Generated/Model/Internal/MarshallTransformations. You can see for there is a separate marshaller or unmarshaller, depending on the direction the type is used, for each .NET type. These are not meant to be called directly and our tied to the SDK's internal logic but you can see what we are doing.

I'm still not fully following what you are trying to do so I'm not sure this code will be helpful.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.