Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime releases management #304

Open
16 of 22 tasks
anaelleltd opened this issue May 8, 2024 · 13 comments
Open
16 of 22 tasks

Runtime releases management #304

anaelleltd opened this issue May 8, 2024 · 13 comments

Comments

@anaelleltd
Copy link
Collaborator

anaelleltd commented May 8, 2024

Overview:

This document aims to summarise proposed/existing options for formalising the process by which:

  • the Fellowship actively consults and collaborates with ecosystem teams
  • scheduled runtime upgrades account for potential breaking changes
  • new bugs and recurring issues identified by ecosystem teams are prioritised for fixes

Considerations:

Strategies:

Note: A ☑ indicates that the item is already In Progress.

  • Draft an outline of planned core developments with Polkadot 2.0 Roadmap
  • Provide changelogs with contextual information for PRs with PRDoc
  • Set up custom assignments and notifications for PR reviews with Review bot
  • Highlight breaking changes more prominently in the changelogs
  • Standardise the release process with a summary doc of key stages
  • Create a distribution list of builders for targeted notifications on Element, Telegram, email, Github, and more
  • Set up a release calendar for communications with ecosystem teams
  • Track the schedules of minor and patch releases in Projects [Internal]
  • Fast track parachains' replacement of their head state and/or wasm with Unbrick collective
  • Set up a comprehensive bank of tests for runtime upgrades with Ecosystem test environment
  • Make it mandatory that any Whitelisted Caller proposal include concrete steps (followable by any Rust engineer) on how to verify it, pointing to existing documentation for the basics
  • Create a clone network to be used to test runtime/node interactions with Polkadot Doppelganger
  • Integrate Doppelganger in Subsquare so that all Whitelisted caller proposals must (by default) have an associated test, which got verified by developers according to a checklist; with the test and test result shown in the UI
  • Inform ecosystem builders about storage item changes with "Metadata diff tool"
  • Others/TBC

Contributors:

  • Parity: Ecosystem success, Engineering, Automation
  • Fellowship: Secretary, Members, Fellows, and Architects
  • Ecosystem: Parachain teams, dapp builders, Unbrick Collective, JUST OpenSource, OpenGov.Watch

Contacts:

This table can be edited & copied/pasted as a comment on scheduled runtime upgrades to ensure that relevant teams are contacted based on proposed/queued PRs.
The contact details for each team are available here.

- Teams Breaking changes Notification timelines
CEXes Binance, Bybit, Coinbase, Gateio, Kraken, Kucoin, OKX, Upbit, etc. Transaction encoding, Event/Error encoding 4+ weeks
Wallets Enkrypt, Fearless, Infinity, Kampela/Ledger, Mimir, Multix, Nova/Polkadot Vault, Polkadot-JS, PolkaGate, PolkaSafe, Polkawallet, Spekter, Subwallet, Talisman/Signet, Tangem, etc. Transaction encoding 4+ weeks
Explorers Polkascan, Statescan, SubID, Subscan XCM and storage format 3+ weeks
User Interfaces Delegation dashboard, DoTreasury/Subsquare, Kodadot, NOMI, Polkadot-JS Apps, Polkassembly/Townhall, Proof of Chaos, Singular, Staking dashboard Extrinsic usage, XCM and storage format 3+ weeks
Parachains Polkadot & Kusama polkadot-sdk (especially migrations) and XCM Ad hoc
Live networks Aleph Zero, Bittensor, Cere Network, ChainFlip, Joystream, Ternoa, Vara, etc. polkadot-sdk (especially migrations) and XCM Ad hoc

Last updated: 15th October 2024

@anaelleltd
Copy link
Collaborator Author

Please share your suggestions or recommendations so that the document above can be made as comprehensive as possible.

@joepetrowski
Copy link
Contributor

Some suggestions:

  • CEXs: Add Event/Error encoding
  • DEXs: I don't quite get what you are grouping here. A lot of them are parachains and should just be your "Parachains" category. I would delete this row.
  • Wallets: Transaction encoding
  • Explorers: AFAIK these are pretty resilient to breaking changes at this point, since we've added types to metadata.
  • OpenGov dApps, NFTs, and Dashboards can all be collapsed into one category: "User Interfaces".
  • Parachains: Breaking changes of concern should be polkadot-sdk breaking changes (especially requiring migrations) and XCM breaking changes.
  • Bridges: Would probably delete. Some of them (P<>K, Snow) are part of the release itself. Others are parachains and thus fit into the "Parachains" category.
  • System Chains: Delete. They are part of the release. Who will the Fellowship notify? Itself?

@anaelleltd
Copy link
Collaborator Author

Thank you for these clarifications and corrections, @joepetrowski.
I have included them in the main document.

@bkchr
Copy link
Contributor

bkchr commented May 10, 2024

One thing that I discussed today with @eskimor is the introduction of some sort of "metadata diff tool". This should help with informing people if for example storage items change. This tool could run for each pr merged to main and later for the release itself. There should exists "client" that can read these diffs and find out if a storage item that your app is using while change. By doing it per pull request the teams could get notified before the actual release is happening. The teams would need to run this tool maybe once per night to get notified.

@josepot you are already generating interfaces for the runtimes using metadata, not sure if you could maybe easily figure out what apis a user is using and based on this determine the storage item.

@josepot
Copy link

josepot commented May 10, 2024

@josepot you are already generating interfaces for the runtimes using metadata,

yes, of course! All of the generated code comes directly from the metadata.

not sure if you could maybe easily figure out what apis a user is using and based on this determine the storage item.

In a way our type system already does this for our users, because if the types generated from the next runtime don't match, then the typescript compiler will let our users know what has changed.

One thing that I discussed today with @eskimor is the introduction of some sort of "metadata diff tool". This should help with informing people if for example storage items change

We could easily use the tools that we have built to generate a diff tool like this. We have already built similar tools that we use internally to detect common types across different chains, so that we can give them some friendly names for the developers.

So, yeah, we could totally build a "metadata diff tool", for sure! How urgent/important is this?

The teams would need to run this tool maybe once per night to get notified.

Wouldn't it be possible/desirable to use chopsticks to get the resulting metadata that will take place after a certain referenda gets enacted, and then run the diff tool against the resulting metadatas of those referendas? 🤔

@bkchr
Copy link
Contributor

bkchr commented May 10, 2024

So, yeah, we could totally build a "metadata diff tool", for sure! How urgent/important is this?

Not a requirement that you write it. The general idea here would also be that teams are informed before the stuff is on chain, so the apps do not break in production. Better they are informed before, make the code work with both version and then be ready when the runtime upgrade is live.

@bkchr
Copy link
Contributor

bkchr commented Jul 1, 2024

We also should highlight breaking changes more prominently in the changelogs. Basically it should be easier to find out when a change breaks something, like #337 which was made not clear that it breaks the transaction encoding.

@bkchr
Copy link
Contributor

bkchr commented Jul 1, 2024

We should start simple by including the following information in the changelog:

  • Does it break the transaction encoding?
  • List all the storage keys that do not exist anymore or which type has changed.

@bkchr
Copy link
Contributor

bkchr commented Jul 1, 2024

We should create multiple communication channels for pushing information around upcoming releases. When we are aware that breaking changes will hit a release, we should start to push out information and the closer we get to the actual enactment of the update the more spammy we should get. Better to over deliver and to annoy people, than people to miss the information. We should support Element, TG, email, github, maybe more? A release calendar.

@jak-pan
Copy link

jak-pan commented Jul 1, 2024

Thanks 👍🏻 I would maybe add that linking to PRs and showing off the integration tests that changed in the breaking change PRs will get a long way to quickly evaluate the required effort. Alternatively tips like -> Update your .js libs would help. Having calendar to see it visually would be epic.

@anaelleltd
Copy link
Collaborator Author

anaelleltd commented Jul 2, 2024

@bkchr @jak-pan @SBalaguer
I have summarised the requirements for a runtime release calendar here
Please let me know if I missed anything important.

@anaelleltd
Copy link
Collaborator Author

Further suggestions from @eskimor

image

@Imod7
Copy link
Contributor

Imod7 commented Sep 30, 2024

In the past, subwasm was used during the release process to generate a comparison between different runtimes. Is there any plan to use it again or maybe another tool that generates the same or similar report? Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants