tuned moe configs v2

[divakar-amd]

Update the fused moe config.json files. These config files utilizes all the available Triton kernel parameters for tuning. Used by both Prefill and Decode fused_moe kernels.

[hthangirala]

As per our discussion today, update this PR with the following:

1. Tuning script used to generate this config.
2. Collect perf with tuning using latest vllm (use Matt's docker to save time: rocm/pytorch-private:vllm0.4.3_ROCm6.1_exec_dashboard_pretuned_0604)
3. Collect perf on H100 perf numbers to compare:
   • Using upstream which defaults to ray
   • Using upstream with torchrun. (Check with Matt on how he got our near-upstream vllm to run with torchrun)

[shajrawi]

I will leave the final confirmation for Hari - but we are good to go as far as the state of our master branch is concerned.

[hthangirala]

Couple items we discussed to complete this PR:

1. Post Mixtral improvement with this PR using TP=1, 2, 4, 8
2. Confirm moe tuning using bfloat16 provides similar moe kernel perf (time) if using the tuning config from this PR (done using float16).

[divakar-amd]

Docker used: rocm/pytorch-private:vllm0.4.3_ROCm6.1_exec_dashboard_pretuned_0605

Table below shows the mixtral mi300 timings (seconds) with v1 & v2 (this PR) moe tuned configs

Calculating % gain observed from the above table

The charts below show timings with v1 & v2 moe tuned configs

The table below shows TP=8 fp16 vs bf16 numbers using the v2 config. The bf16 numbers were similar to that obtained when tuned with bf16 datatype.

[hthangirala]

Thanks for the measurements!

[divakar-amd]

Fixed. Reverted the revert. Added separate commit for init files fix.