A cli tool to easily spawn ephemeral Polkadot/Substrate networks and perform tests against them.
This project is still in early stage and very much a work in progress. More features will be added, docs may be missing or outdated and api/config may change.
NOTE: polkadot-collator has recently been renamed polkadot-parachain.
Zombienet aims to be a testing framework for Substrate based blockchains, providing a simple cli
tool that allows users to spawn and test ephemeral networks. The assertions used in the tests can
include on-chain storage, metrics, logs and custom javascript scripts that interact with the chain.
To make it easy to define those, zombienet has a natural language built-in allowing developers to
write tests as smooth as possible.
Internally zombienet is a javascript library, designed to run on Node.js and support different
backend providers to run the nodes, at this moment kubernetes, podman and native are
supported.
Zombienet releases are available in github. Each one provides an executable for both linux and
macos created with pkg and allows to run zombienet cli
without having Node.js installed but each provider defines its own requirements (e.g.
k8s, podman).
Note: Currently, it is only possible to use podman for Zombienet users on Linux machines.
Although podman comes with support for macOS, it is done using an internal VM and the Zombienet provider code expects podman to be running natively.
After you have downloaded zombienet-macos-arm64 or zombienet-macos-x64, you will need to:
- Move the binary to your working directory.
- Rename the binary to just
zombienetwithout anymacos-<version>extension for convenience. - Enable the binary to be executable:
chmod +x ./zombienet
- Remove the binary from quarantine:
xattr -d com.apple.quarantine ./zombienet
Then you should be able to access the binary:
./zombienet helpIf you have Node.js, you can install zombienet locally via NPM:
npm i @zombienet/cli -gThen you should be able to access the zombienet command:
zombienet help
At the moment Zombienet only works with local chains (e.g. rococo-local, polkadot-local, etc).
Zombienet should work with any k8s cluster (e.g GKE,
docker-desktop, kind)
but you need to have kubectl installed to interact with your cluster.
Also, you need permission to create resources (e.g namespaces, pods and cronJobs) in the
target cluster.
Zombienet project has it's own k8s cluster in GCP, to use it please ping
Javier(@javier:matrix.parity.io) in element to gain access and steps to use.
Zombienet supports Podman rootless as provider, you only need to have
podman installed in your environment to use and either set in the network file or with the
--provider flag in the cli. Podman for zombienet is currently only supported for Linux machines.
This is mostly related to paths and directories used by
store configuration (chain-spec) and the data directory.
Zombienet native provider allows you to run the nodes as a local process in your environment. You
only need to have the binaries used in your network (e.g polkadot or polkadot-parachain).
To use it either set in the network file or with the --provider flag in the cli.
NOTE: The native provider only uses the command config for nodes/collators, both relative
and absolute paths are supported. You can use default_command config to set the binary to spawn
all the nodes in the relay chain.
Alternative: You can set the command to the binary directly if is available in your PATH.
With k8s zombienet use "Prometheus operator" (if it is available) to offload the
monitoring/visibility layer, so only the network's pods are deployed by zombienet.
With podman zombienet deploys a couple of extra pods to add a layer of monitoring/visibility to
the running network. In particular pods for prometheus, tempo and grafana are deployed. Also,
grafana is configured to have prometheus and tempo as datasource.
To access those services you can find the url in the output of zombinet:
Monitor: prometheus - url: http://127.0.0.1:34123
Monitor: tempo - url: http://127.0.0.1:34125
Monitor: grafana - url: http://127.0.0.1:41461Note: Grafana is deployed with the default admin access.
Once the network is stopped, by ctrl+c on a running spawn or by finishing the test, these pods are
removed with the rest of the pods launched by zombienet.
Native provider doesn't run any extra layer/process at the moment.
For this example we will use the macos version of the executable
❯ ./zombienet-macos
Usage: zombienet [options] [command]
Options:
-c, --spawn-concurrency <concurrency> Number of concurrent spawning process to launch, default is 1
-p, --provider <provider> Override provider to use (choices: "podman", "kubernetes", "native")
default: kubernetes
-l, --logType <logType> Type of logging on the console - defaults to 'table' (choices: "table", "text", "silent")
-d, --dir <path> Directory path for placing the network files instead of random temp one (e.g. -d /home/user/my-zombienet)
-f, --force Force override all prompt commands
-m, --monitor Start as monitor, do not auto cleanup network
-h, --help display help for command
Commands:
spawn <networkConfig> [creds] Spawn the network defined in the config
test <testFile> Run tests on the network defined
version Prints zombienet version
help [command] display help for commandOne of the goals of zombienet is to easily spawn ephemeral networks, providing a simple but
powerful cli that allows you to declare the desired network in toml or json format. You can
check the definition spec to view the available options.
A minimal configuration example with two validators and one parachain:
[settings]
timeout = 1000
[relaychain]
default_image = "paritypr/polkadot-debug:master"
chain = "rococo-local"
[[relaychain.nodes]]
name = "alice"
[[relaychain.nodes]]
name = "bob"
[[parachains]]
id = 100
[parachains.collator]
name = "collator01"
image = "paritypr/colander:master"
command = "adder-collator"Then you can spawn the network by running the following command:
❯ ./zombienet-macos spawn --provider native examples/0001-small-network.tomlNote that the command expects two binaries polkadot and adder-collator to be installed on your system. See further down for how to get them.
You can follow the output of the steps to spawn the network and once the network is launched a
message with the nodes information like this one is shown
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Network launched 🚀🚀 │
├─────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Namespace │ zombie-72a1e2ffad0ad73167061bbd560e0766 │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Provider │ native │
├─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Node Information │
├─────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Name │ alice │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Direct Link │ https://polkadot.js.org/apps/?rpc=ws://127.0.0.1:45589#/explorer │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Prometheus Link │ http://127.0.0.1:44107/metrics │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Log Cmd │ tail -f /tmp/zombie-85391d4649f2829bb26b30d6c0328bcb_-15819-BNFoSs5qusWH/alice.log │
├─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Node Information │
├─────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Name │ bob │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Direct Link │ https://polkadot.js.org/apps/?rpc=ws://127.0.0.1:46459#/explorer │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Prometheus Link │ http://127.0.0.1:43831/metrics │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Log Cmd │ tail -f /tmp/zombie-85391d4649f2829bb26b30d6c0328bcb_-15819-BNFoSs5qusWH/bob.log │
├─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Node Information │
├─────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Name │ collator01 │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Direct Link │ https://polkadot.js.org/apps/?rpc=ws://127.0.0.1:42607#/explorer │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Prometheus Link │ http://127.0.0.1:38281/metrics │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Log Cmd │ tail -f /tmp/zombie-85391d4649f2829bb26b30d6c0328bcb_-15819-BNFoSs5qusWH/collator01.log │
├─────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Parachain ID │ 100 │
└─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
│ ChainSpec Path │ /tmp/zombie-85391d4649f2829bb26b30d6c0328bcb_-15819-BNFoSs5qusWH/rococo-local-100.json │
└─────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
Both the prometheus and the node links are accessible in your local machine to get the metrics
or connect to the node.
Zombienet can also make replacements in the network config using environment variables. To define
a replacement you need to use the {{ENV_VAR_NAME}} syntax.
For example, from the previous example but using env variables could be:
[relaychain]
default_image = "{{ZOMBIENET_INTEGRATION_TEST_IMAGE}}"
chain = "rococo-local"
[[relaychain.nodes]]
name = "alice"
[[relaychain.nodes]]
name = "bob"
[[parachains]]
id = 100
add_to_genesis = false
[parachains.collator]
name = "collator01"
image = "{{COL_IMAGE}}"
command = "adder-collator"Then you can export the needed values before you run the command to spawn the network again:
❯ export ZOMBIENET_INTEGRATION_TEST_IMAGE=docker.io/paritypr/polkadot-debug:master
❯ export COL_IMAGE=docker.io/paritypr/colander:master
./zombienet-macos spawn examples/0001-small-network.tomlYou can teardown the network (and cleanup the used resources) by terminating the process (ctrl+c).
The other goal of zombienet is to provide a way to perform test/assertions against the spawned
network, using a set of natural language expressions that allow you to make assertions based on
metrics, logs and some built-in function that query the network using polkadot.js. Those
assertions should be defined in a .zndsl test, and the dsl (Domain Specific Language) and format is documented in
here.
The following is a small example to spawn a network (using the previous simple network definition) and assert that:
- Both
nodesare running - The defined
parachainis registered - The defined
parachainis producing blocks and produced at least 10 within 200 seconds.
Description: Simple Network Smoke Test
Network: ./0001-small-network.toml
Creds: config
alice: is up
bob: is up
alice: parachain 100 is registered within 225 seconds
alice: parachain 100 block height is at least 10 within 200 secondsOther examples are provided in the examples directory.
- Node.js if you are not using the self contained linux or macos releases.
- Kubernetes cluster to use
kubernetestarget (kubectlcommand installed). - Podman to use
podmantarget.
You need first to clone this repository and run:
❯ cd zombienet/javascript
❯ npm i && npm run buildFor an easier and faster setup of your local environment, run:
❯ cd zombinet/javascript
❯ npm i && npm run zombie -- setup <binaries>This allows to use the setup script, making everything ready for a ZombieNet dev environment.
You can use the following arguments:
--help shows the different options and commands for using the Zombienet CLI.
--binaries or -b: enables providing the binaries that you want to be downloaded and installed during the setup. Possible options: all, polkadot, polkadot-parachain. Note: Downloading polkadot will automatically download also the binaries of polkadot-prepare-worker, polkadot-execute-worker. Since Polkadot v1.0 all 3 binaries are needed for the node to run as a validator;
For example:
❯ cd zombinet/javascript
❯ npm i && npm run zombie -- setup polkadot polkadot-parachainNote: If you are using macOS please clone the polkadot-sdk repo and run it locally. At the moment there is no
polkadotbinary for MacOs.
The command above will retrieve the binaries provided and try to download and prepare those binaries for usage.
At the end of the download, the setup script will provide a command to run in your local environment in order to add the directory where the binaries were downloaded in your $PATH var, for example:
Please add the dir to your $PATH by running the command: export PATH=/home/<user>/zombienet/dist:$PATHYou can build it from source like this
❯ git clone git@github.com:paritytech/polkadot-sdk.git
❯ cd polkadot-sdk
❯ cargo build --profile testnet -p test-parachain-adder-collator
❯ export PATH=$(pwd)/target/testnet:$PATHWith the above steps completed, the zombienet CLI is ready to run:
❯ cd zombinet/javascript
❯ npm run zombie
Usage: zombienet [options] [command]
Options:
-c, --spawn-concurrency <concurrency> Number of concurrent spawning process to launch, default is 1
-p, --provider <provider> Override provider to use (choices: "podman", "kubernetes", "native")
-l, --logType <logType> Type of logging - defaults to 'table' (choices: "table", "text", "silent")
-d, --dir <path> Directory path for placing the network files instead of random temp one
(e.g. -d /home/user/my-zombienet)
-f, --force Force override all prompt commands
-h, --help display help for command
Commands:
spawn [options] <networkConfig> [creds] Spawn the network defined in the config
test <testFile> [runningNetworkSpec] Run tests on the network defined
setup [options] <binaries...> Setup is meant for downloading and making dev environment of ZombieNet ready
convert <filePath> Convert is meant for transforming a (now deprecated) polkadot-launch configuration to zombienet configuration
version Prints zombienet version
help [command] display help for commandWith nix run nix run github:paritytech/zombienet or add inputs.zombienet.url = "github:paritytech/zombienet"; to flake.
Below can be found some of the projects that are currently using Zombienet as integration or as a testing framework:
In Parity:
- Cumulus
- Polkadot in the testing pipeline;
- Substrate
In the Polkadot ecosystem:
- Acurast
- Composable via
nix run "composable#devnet-picasso" - Gossamer
- Oak/Turing/Neumann
- Hydradx
- InvArch
- Mangata
- Manta/Phala
- Moonbeam
- T3rn
This project takes inspiration and some patterns from polkadot-launch and simnet.