Why S3 service model is not automatically generated?

[wlandgraf]

Hi all,

I'm checking the service models and I noticed that the files for s3 service model are mostly in the Custom folder, not in Generated folder. However, the content of the files there are similar to the generated ones (Response, Request, Marshaller, etc.)

Why is it structured that way? Why the metadata is not configured to generate the S3 model files automatically?

At least, I assume that at some point those custom S3 files were generated automatically and then customized? If yes, why?

[ashishdhingra]

@wlandgraf There are historical reasons on why most of the code for AWSSDK.S3 package is not auto-generated, instead hand-crafter. I would defer this question to @normj to provide right guidance.

CC @normj

[wlandgraf]

Thank you @ashishdhingra.
Yes, I imagine it's for historical reasons, but maybe those files were written even before the generator?
What I wonder is how much the existing custom code differ from what the generator would produce, because the custom code looks very similar to generated one, specially the model of structures and the marshallers/unmarshallers.

Looking forward for more info from your colleagues.

[normj]

Hi @wlandgraf. Yes the S3 codebase predates our code generation which is why it is hand coded. We have tried to switch over some of the pieces to code generation like the service clients. Our custom code is similar to what would be generated but given we coded it before there was an S3 model there are many cases where named operations and properties are different then was later named when S3 created a model. Also there are cases where we model something as full object hierarchy but the model models it all as a string.

For efficiency and consistency sake we would like to get S3 as part of the code generation but given how widely used S3 is, making breaking changes to S3 is not practical. So then we need to invest a lot into new customization hooks into our generator to have the generator generate from the model and customization files the same public API we have today. We did that many years ago for EC2 which used to be handcoded and it was a lot of work. We have debated many times about doing the same for S3 but we haven't been able to find the time to do it.

[wlandgraf]

Thank you very much for the detailed explanation, @normj.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.