Cadence Testing Framework Major Uplift

[m-Peter]

Cadence Testing Framework

Grant category

Please select one:

• Open source maintenance
• Developer tools / services
• Educational material

Description

The Cadence Testing Framework has come a long way, after the completion of #148. It has been successfully used as the testing suite of https://github.com/onflow/hybrid-custody, enhancing the project's CI pipeline and DevEx. A number of other projects have been using the framework as well, including the FungibleToken and NonFungibleToken standards plus some other community projects.

Problem statement

After these first two iterations, we have identified some missing features which will make the framework feature-complete.
Up until now, there have been two types of testing: unit & integration tests. These two required a very different approach of testing, which resulted in some inconsistencies and a lack of features for both testing types. Unit tests were very limited in testing very primitive contracts, and integration tests could not make use of deployed contract types, which is one of the most needed features.
This lack of features is described here: onflow/cadence-tools#209

Proposed solution

To overcome all these limitations, we will unify the backend environment for both unit & integration tests. The emulator blockchain will no longer be exposed to the testing framework, as it will be just a singleton. Test code will have to deploy a contract prior to using it. The accounts where contracts are to be deployed, will be specified in the familiar flow.json config file, in a newly-introduced network (testing). We have borrowed an idea from the Flow Playground, and will bootstrap the framework with 10 predefined accounts, for contract deployment. More details can be found in this post: https://forum.onflow.org/t/major-uplift-for-cadence-testing-framework/5232

Impact

Provide test / development parity, by allowing developers to write unit/integration tests in Cadence (https://12factor.net/dev-prod-parity). Developers will also be able to measure how well their tests cover their contracts, and gain sufficient confidence before deploying to testnet, and more importantly on mainnet.

Milestones and funding

Milestone	Deliverables	Timeline	Risks	USD proposal
1 - Implementation	Merge onflow/cadence-tools#210 & onflow/flow-cli#1227	4 weeks	-	5000
2 - Adoption	Update 5 projects to use the new testing API	2 weeks	-	2500

Total funding proposed: 7500 $

Team

Name	Role	Bio	Contact
Ardit Marku	Back-end Engineer	A seasoned back-end engineer with a focus on problem solving and designing robust software systems. Over 9 years of experience building high-traffic search engines, marketplaces, exchanges and in-house analytics solutions. Led cross-functional teams to deliver both web and mobile apps, participating in all steps of the software development lifecycle. Open source lover and advocate.	markoupetr@gmail.com m_peter

[m-Peter]

An example of unit testing the built-in Crypto contract, in the cadence repository: onflow/cadence#2856

[m-Peter]

Status Update:

1st Milestone:

• [test] Unify backend environment for Unit & Integration tests cadence-tools#210 merged
• Update test command with latest features from Cadence testing framework flow-cli#1227 merged

2nd Milestone:

• Add tests for RandomBeaconHistory contract flow-core-contracts#389
• Update Cadence tests to latest API features flow-nft#192
• Update Cadence tests to latest API features flow-ft#150
• Update Cadence tests to latest API features hybrid-custody#161
• Update tests for Crypto contract to the latest testing framework API cadence#2856

As soon as a new flow-cli release is made available, I can proceed with the above 5 PRs which migrate the testing suite to the new API.

cc @franklywatson

[m-Peter]

Besides the Flow core repositories, I am adding one more migration from a community-driven project:

• Update Cadence tests to latest API features and include them in unit-tests workflow green-goo-dao/flow-utils#22

[m-Peter]

Hey @franklywatson
The 2nd milestone is also completed. The testing suite for 5 Flow core projects has been migrated. The relevant PRs are all merged. I have migrated another community project 🙏
I believe we can mark the entire grant as completed 🎉

[franklywatson]

Great work. Thanks!