Rate of breaking changes is painful

[GuyPozner]

Hello,

The rate at which hera-workflows introduces breaking changes makes it very hard to work with. Do you have a plan to slow down? introduce backward compatibility? communicate breaking changes in advance?

[flaviuvadan]

Hello @GuyPozner! Thanks for submitting this. This is a huge topic in the open-source community, and for Hera, and I really appreciate you bringing it up. I will try my best to offer an explanation of why these changes were required, how we plan to slow down, not introduce such breaking changes anymore, and some history.

I used to be the only maintainer for a bit of time, and Hera is my first open source project that is actually used. During that time, it was challenging to imagine all the way people would use Hera in, and their specific Argo/K8s setup that would require specific features on Hera’s side. An example of this was back when we moved from v2 to v3 (I think), because of an SSL setting that required a breaking change. So, it was my own lack of experience with different Argo setups that forced Hera in this direction, as I did not effectively predict all the feedback that came from the community - I am fully responsible for this. In addition, the way Hera approached making workflow submission easier actually limited usage and accessibility. By introducing specific abstractions and merging concepts (such as task + template and workflow + dag) Hera took away users’ ability to take advantage of the full feature set of Argo. To address some of these limitations and offer users full access to Argo’s feature set, Hera needed to change its approach... development was becoming a bottleneck - the focus was on reimplementing code/concepts rather than new features that would improve the user quality of life. To be perfectly transparent, I was very wary that a new version would be disappointing to users, because I do understand the pain of migrating versions, particularly when the changes are not only “change param X to Y and we’re fine”. However, after much deliberation, I accepted that this is a necessary change in Hera’s evolution, hence v5. With v5, we have:

• full feature parity with Argo Workflows and Events. This was not the case before as contributors had to reimplement concepts. If you look through recent issues you will see users mentioning that, for instance, not all volumes are available, volumes are hard to extend/change, pod GC is not available, etc. Now all of these are available/solved!
• readable service code. Compare this to the auto-generated code of OpenAPI, which was challenging to “click through” to understand what it did. Yes, there are some small challenges with auto-generating the service ourselves as well, such as conflicting imports, but they are much easier to see/understand
• now the focus can be fully dedicated to improving user workflows. Previously we had to reimplement core concept of Argo. Now, when they are not available as syntactic sugar, users still have the liberty to use the autogenerated code in a clear manner, and contributions can be focused on bridging the gap between Hera and the auto-generated models, making those models more accessible, introducing features that better support the downstream development of platforms, etc.

To address the questions specifically:

• Do you have a plan to slow down?
  • yes. Given V5 has feature parity we can now focus on exclusively building on top of/extending it, rather than keep on changing core elements
• introduce backward compatibility?
  • the focus will be exclusively backwards compatibility in whatever is introduced from now on. This was the focus before as well, within the context of a major version
  • Hera also has a hera/v4 branch, which is still maintained for patches and necessary features and bug fixes on V4
• communicate breaking changes in advance?
  • I feel like the best way to address this is to learn from you - how would you have best received information on the upcoming release? We have a Slack channel in CNCF where we had some discussions on the topic. There was an issue in the repo that outlined what’s coming in V5 (served as a dev checklist). I also gave an Argo Workflows Community presentation on v5, twice. Would it help to have a short blog release? Would it help to have a Twitter post? I am very open to any suggestion that will help you and your team have more awareness of what’s coming to Hera! I know you are using Hera and have contributed to it in the past, so I care for announcing upcoming releases better so you and your team can be as informed as possible. I’d love to hear your feedback here.

I am also not the only long-term maintainer now - @samj1912 and @elliotgunton are now officially maintainers as well. They have been pivotal in the thinking behind v5, are big users of Hera in their organization, want Hera to have a stable version going forward in order to decrease toil on users who have to migrate versions, and want Hera to be focused on QOL features rather than reimplementation of concepts.

Again, I want to emphasize how much I appreciate this question. I hope some of the explanations above help offer some context behind the decision to release v5.

[flaviuvadan]

@GuyPozner one actionable take away in the realm of improving announcements/transparency around changes is the use of Discussions. So, we started using Discussions as well!

[flaviuvadan]

@GuyPozner I'd love to hear any other feedback you have on this discussion! In the mean time I am closing this as it's been open for a while. Feel free to re-open if you'd like to continue discussing the topic. Again, thank you for bringing this up :)