Skip to content

Commit

Permalink
Merge branch 'master' into ao-account-for-enact-candidate-weight
Browse files Browse the repository at this point in the history
* master: (130 commits)
  add elastic scaling MVP guide (#4663)
  Send PeerViewChange with high priority (#4755)
  [ci] Update forklift in CI image (#5032)
  Adjust base value for statement-distribution regression tests (#5028)
  [pallet_contracts] Add support for transient storage in contracts host functions (#4566)
  [1 / 5] Optimize logic for gossiping assignments (#4848)
  Remove `pallet-getter` usage from pallet-session (#4972)
  command-action: added scoped permissions to the github tokens (#5016)
  net/litep2p: Propagate ValuePut events to the network backend (#5018)
  rpc: add back rpc logger (#4952)
  Updated substrate-relay version for tests (#5017)
  Remove most all usage of `sp-std` (#5010)
  Use sp_runtime::traits::BadOrigin (#5011)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  ...
  • Loading branch information
ordian committed Jul 18, 2024
2 parents 329a390 + 0db5092 commit e6978fd
Show file tree
Hide file tree
Showing 1,883 changed files with 43,897 additions and 27,753 deletions.
7 changes: 7 additions & 0 deletions .config/taplo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,10 @@ keys = ["build"]

[rule.formatting]
reorder_arrays = false

[[rule]]
include = ["Cargo.toml"]
keys = ["workspace.dependencies"]

[rule.formatting]
reorder_keys = true
33 changes: 33 additions & 0 deletions .forklift/config-gitlab.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
[compression]
type = "zstd"

[compression.zstd]
compressionLevel = 3

[general]
jobNameVariable = "CI_JOB_NAME"
jobsBlackList = []
logLevel = "warn"
threadsCount = 6

[cache]
extraEnv = ["RUNTIME_METADATA_HASH"]

[metrics]
enabled = true
pushEndpoint = "placeholder"

[metrics.extraLabels]
environment = "production"
job_name = "$CI_JOB_NAME"
project_name = "$CI_PROJECT_PATH"

[storage]
type = "s3"

[storage.s3]
accessKeyId = "placeholder"
bucketName = "placeholder"
concurrency = 10
endpointUrl = "placeholder"
secretAccessKey = "placeholder"
10 changes: 3 additions & 7 deletions .forklift/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,7 @@ job_name = "$CI_JOB_NAME"
project_name = "$CI_PROJECT_PATH"

[storage]
type = "s3"
type = "gcs"

[storage.s3]
accessKeyId = "placeholder"
bucketName = "placeholder"
concurrency = 10
endpointUrl = "placeholder"
secretAccessKey = "placeholder"
[storage.gcs]
bucketName = "parity-ci-forklift"
36 changes: 36 additions & 0 deletions .github/actions/set-up-gh/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: 'install gh'
description: 'Install the gh cli in a debian based distro and switches to the PR branch.'
inputs:
pr-number:
description: "Number of the PR"
required: true
GH_TOKEN:
description: "GitHub token"
required: true
outputs:
branch:
description: 'Branch name for the PR'
value: ${{ steps.branch.outputs.branch }}
runs:
using: "composite"
steps:
- name: Instal gh cli
shell: bash
# Here it would get the script from previous step
run: |
(type -p wget >/dev/null || (apt update && apt-get install wget -y))
mkdir -p -m 755 /etc/apt/keyrings
wget -qO- https://cli.github.com/packages/githubcli-archive-keyring.gpg | tee /etc/apt/keyrings/githubcli-archive-keyring.gpg > /dev/null
chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list > /dev/null
apt update
apt install gh -y
git config --global --add safe.directory '*'
- run: gh pr checkout ${{ inputs.pr-number }}
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.GH_TOKEN }}
- name: Export branch name
shell: bash
run: echo "branch=$(git rev-parse --abbrev-ref HEAD)" >> "$GITHUB_OUTPUT"
id: branch
Binary file added .github/command-screnshot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
200 changes: 200 additions & 0 deletions .github/commands-readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# Running commands

Command bot has been migrated, it is no longer a comment parser and now it is a GitHub action that works as a [`workflow_dispatch`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch) event.

## How to run an action

To run an action, you need to go to the [_actions tab_](https://github.com/paritytech/polkadot-sdk/actions) and pick the one you desire to run.

The current available command actions are:

- [Command FMT](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-fmt.yml)
- [Command Update UI](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-update-ui.yml)
- [Command Sync](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-sync.yml)
- [Command Bench](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-bench.yml)
- [Command Bench All](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-bench-all.yml)
- [Command Bench Overhead](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-bench-overhead.yml)

You need to select the action, and click on the dropdown that says: `Run workflow`. It is located in the upper right.

If this dropdown is not visible, you may not have permission to run the action. Contact IT for help.

![command screenshot](command-screnshot.png)

Each command will have the same two required values, but it could have more.

GitHub's official documentation: [Manually running a workflow](https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow)

### Number of the Pull Request

The number of the pull request. Required so the action can fetch the correct branch and comment if it fails.

## Action configurations

### Bench

Runs `benchmark pallet` or `benchmark overhead` against your PR and commits back updated weights.

Posible combinations based on the `benchmark` dropdown.

- `substrate-pallet`: Pallet Benchmark for Substrate for specific pallet
- Requires `Subcommand` to be `pallet`
- Requires `Runtime` to be `dev`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Target Directory` to be `substrate`
- `polkadot-pallet`: Pallet Benchmark for Polkadot for specific pallet
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `rococo`
- `westend`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Target Directory` to be `polkadot`
- `cumulus-assets`: Pallet Benchmark for Cumulus assets
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `asset-hub-westend`
- `asset-hub-rococo`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `assets`
- Requires `Target Directory` to be `cumulus`
- `cumulus-collectives`: Pallet Benchmark for Cumulus collectives
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be `collectives-westend`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `collectives`
- Requires `Target Directory` to be `cumulus`
- `cumulus-coretime`: Pallet Benchmark for Cumulus coretime
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `coretime-rococo`
- `coretime-westend`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `coretime`
- Requires `Target Directory` to be `cumulus`
- `cumulus-bridge-hubs`: Pallet Benchmark for Cumulus bridge-hubs
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `bridge-hub-rococo`
- `bridge-hub-westend`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `bridge-hub`
- Requires `Target Directory` to be `cumulus`
- `cumulus-contracts`: Pallet Benchmark for Cumulus contracts
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one `contracts-rococo`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `contracts`
- Requires `Target Directory` to be `cumulus`
- `cumulus-glutton`: Pallet Benchmark for Cumulus glutton
- Requires `Subcommand` to be `pallet`
- Requires `Runtime` to be one of the following:
- `glutton-westend`
- `glutton-westend-dev-1300`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `glutton`
- Requires `Target Directory` to be `cumulus`
- `cumulus-starters`: Pallet Benchmark for Cumulus starters
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `seedling`
- `shell`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `starters`
- Requires `Target Directory` to be `cumulus`
- `cumulus-people`: Pallet Benchmark for Cumulus people
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `people-westend`
- `people-rococo`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `people`
- Requires `Target Directory` to be `cumulus`
- `cumulus-testing`: Pallet Benchmark for Cumulus testing
- Requires `Subcommand` to be one of the following:
- `pallet`
- `xcm`
- Requires `Runtime` to be one of the following:
- `penpal`
- `rococo-parachain`
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- Requires `Runtime Dir` to be `testing`
- Requires `Target Directory` to be `cumulus`

### Bench-all

This is a wrapper to run `bench` for all pallets.

Posible combinations based on the `benchmark` dropdown.

- `pallet`: Benchmark for Substrate/Polkadot/Cumulus/Trappist for specific pallet
- Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$`
- `substrate`: Pallet + Overhead + Machine Benchmark for Substrate for all pallets
- Requires `Target Directory` to be `substrate`
- `polkadot`: Pallet + Overhead Benchmark for Polkadot
- Requires `Runtime` to be one of the following:
- `rococo`
- `westend`
- Requires `Target Directory` to be `polkadot`
- `cumulus`: Pallet Benchmark for Cumulus
- Requires `Runtime` to be one of the following:
- `rococo`
- `westend`
- `asset-hub-kusama`
- `asset-hub-polkadot`
- `asset-hub-rococo`
- `asset-hub-westend`
- `bridge-hub-kusama`
- `bridge-hub-polkadot`
- `bridge-hub-rococo`
- `bridge-hub-westend`
- `collectives-polkadot`
- `collectives-westend`
- `coretime-rococo`
- `coretime-westend`
- `contracts-rococo`
- `glutton-kusama`
- `glutton-westend`
- `people-rococo`
- `people-westend`
- Requires `Target Directory` to be `cumulus`

### Bench-overhead

Run benchmarks overhead and commit back results to PR.

Posible combinations based on the `benchmark` dropdown.

- `default`: Runs `benchmark overhead` and commits back to PR the updated `extrinsic_weights.rs` files
- Requires `Runtime` to be one of the following:
- `rococo`
- `westend`
- Requires `Target directory` to be `polkadot`
- `substrate`: Runs `benchmark overhead` and commits back to PR the updated `extrinsic_weights.rs` files
- Requires `Target directory` to be `substrate`
- `cumulus`: Runs `benchmark overhead` and commits back to PR the updated `extrinsic_weights.rs` files
- Requires `Runtime` to be one of the following:
- `asset-hub-rococo`
- `asset-hub-westend`
- Requires `Target directory` to be `cumulus`

## How to modify an action

If you want to modify an action and test it, you can do by simply pushing your changes and then selecting your branch in the `Use worflow from` option.

This will use a file from a specified branch.
2 changes: 1 addition & 1 deletion .github/env
Original file line number Diff line number Diff line change
@@ -1 +1 @@
IMAGE="docker.io/paritytech/ci-unified:bullseye-1.77.0-2024-04-10-v20240408"
IMAGE="docker.io/paritytech/ci-unified:bullseye-1.77.0-2024-04-10-v202407161507"
2 changes: 2 additions & 0 deletions .github/review-bot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ rules:
- ^\.gitlab/.*
- ^\.config/nextest.toml
- ^\.cargo/.*
- ^\.forklift/.*
exclude:
- ^\.gitlab/pipeline/zombienet.*
type: "or"
Expand All @@ -33,6 +34,7 @@ rules:
- ^docker/.*
- ^\.github/.*
- ^\.gitlab/.*
- ^\.forklift/.*
- ^\.config/nextest.toml
- ^\.cargo/.*
minApprovals: 2
Expand Down
10 changes: 6 additions & 4 deletions .github/scripts/check-workspace.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,12 @@ def check_deps(deps):
if dep_name in all_crates:
links.append((name, dep_name))

if not 'path' in deps[dep]:
broken.append((name, dep_name, "crate must be linked via `path`"))
if name == 'polkadot-sdk':
if not 'path' in deps[dep]:
broken.append((name, dep_name, "crate must use path"))
return
elif not 'workspace' in deps[dep] or not deps[dep]['workspace']:
broken.append((name, dep_name, "crate must use workspace inheritance"))
return

def check_crate(deps):
Expand All @@ -154,8 +158,6 @@ def check_crate(deps):

check_crate(manifest)



links.sort()
broken.sort()

Expand Down
14 changes: 12 additions & 2 deletions .github/scripts/deny-git-deps.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

root = sys.argv[1] if len(sys.argv) > 1 else os.getcwd()
workspace = Workspace.from_path(root)
errors = []

def check_dep(dep, used_by):
if dep.location != DependencyLocation.GIT:
Expand All @@ -27,14 +28,23 @@ def check_dep(dep, used_by):
if used_by in KNOWN_BAD_GIT_DEPS.get(dep.name, []):
print(f'🤨 Ignoring git dependency {dep.name} in {used_by}')
else:
print(f'🚫 Found git dependency {dep.name} in {used_by}')
sys.exit(1)
errors.append(f'🚫 Found git dependency {dep.name} in {used_by}')

# Check the workspace dependencies that can be inherited:
for dep in workspace.dependencies:
check_dep(dep, "workspace")

if workspace.crates.find_by_name(dep.name):
if dep.location != DependencyLocation.PATH:
errors.append(f'🚫 Workspace must use path to link local dependency {dep.name}')

# And the dependencies of each crate:
for crate in workspace.crates:
for dep in crate.dependencies:
check_dep(dep, crate.name)

if errors:
print('❌ Found errors:')
for error in errors:
print(error)
sys.exit(1)
2 changes: 1 addition & 1 deletion .github/workflows/check-prdoc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
merge_group:

env:
IMAGE: docker.io/paritytech/prdoc:v0.0.7
IMAGE: docker.io/paritytech/prdoc:v0.0.8
API_BASE: https://api.github.com/repos
REPO: ${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
7 changes: 0 additions & 7 deletions .github/workflows/check-runtime-migration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,6 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
FORKLIFT_storage_s3_bucketName: ${{ secrets.FORKLIFT_storage_s3_bucketName }}
FORKLIFT_storage_s3_accessKeyId: ${{ secrets.FORKLIFT_storage_s3_accessKeyId }}
FORKLIFT_storage_s3_secretAccessKey: ${{ secrets.FORKLIFT_storage_s3_secretAccessKey }}
FORKLIFT_storage_s3_endpointUrl: ${{ secrets.FORKLIFT_storage_s3_endpointUrl }}
FORKLIFT_metrics_pushEndpoint: ${{ secrets.FORKLIFT_metrics_pushEndpoint }}

jobs:
set-image:
# GitHub Actions allows using 'env' in a container context.
Expand Down
Loading

0 comments on commit e6978fd

Please sign in to comment.