Feature request: configure all-purpose cluster libraries through DAB

[rsayn]

Describe the issue

Since 0.229.0 all-purpose (interactive) clusters can be created via DAB.

With Job clusters, it's pretty straightforward to install a DAB wheel artifact by specifying the libraries for a task executed on that cluster.

With All-purpose clusters this is currently not possible, and the only solution is to perform post-operations with the SDK or APIs to add a library programmatically.

Configuration

bundle:
  name: demo-dab
  databricks_cli_version: 0.231.0

artifacts:
  default:
    type: whl
    build: poetry build
    path: .

resources:
  clusters:
    interactive:
      cluster_name: ${bundle.name} cluster
      data_security_mode: SINGLE_USER
      # [...] cluster config pointing to an all-purpose policy ID
      # these next lines are currently not valid
      libraries:
        - whl: "../dist/*.whl"

Expected Behavior

There should be a way to specify the deployed bundle wheel as a dependency.

Actual Behavior

There's currently no way to specify this behaviour.
The wheel needs to be post-attached to the cluster via the SDK by:

1. Retrieving the cluster's ID
2. Attaching libraries

Note that both steps would greatly benefit from the substitution happening inside DABs - without it, the cluster name and library path have to be inferred somehow.

OS and CLI version

• Databricks CLI v0.231.0
• MacOS

Is this a regression?

No, this is a new feature request

Debug Logs

N/A