Kubernetes: support default scheduler instead of volcano for autoscaling

[d4l3k]

Description

Right now Volcano doesn't handle autoscaling correctly since it won't schedule jobs if there's not enough resources. For single node jobs it would be nice to support the non-volcano K8s batch scheduler so the Pods will be created and can autoscale the cluster.

Detailed Proposal

Add a setting scheduler to the kubernetes scheduler to allow setting schedulerName to "default-scheduler" to use the standard logic.

https://volcano.sh/en/docs/vcjob/#schedulername

There's some prototype work that was done for TPU node prototyping in 66e934c which required the same default scheduler logic

Alternatives

Additional context/links