Allow configuring dedicated volume for ScyllaDB commit log

[matheus2740]

What should the feature do?

While using the helm chart to deploy a cluster, it is currently impossible to use a separate volume for the commitlog, which is a pretty common operation (and highly recommended when using high latency volumes, such as spinning disks).

racks[].storage currently only specifies one volume, which is where both data and commitlog will live, and ideally we would be able to specify another (optional) key for the commitlog,
such as racks[].commitlogStorage, or subdivide the existing key into racks[].storage.data and racks[].storage.commitlog.

racks[].volumes cannot be used for this purpose because it will mount the volume to the entire rack, and not one per individual node.

What is the use case behind this feature?

Separating the commitlog and data directories into different volumes is required if you wish to have acceptable performance while using spinning disks, and also can be an important strategy to maximize throughput while using networked storage such as EBS.
Since it is hard/impossible to use local disks with helm/k8s, this option should be readily available since the vast majority of users will be using networked storage, and separating the commitlog can greatly improve the cluster performance.

Anything else we need to know?

Briefly discussed on slack:
https://scylladb-users.slack.com/archives/CFFVC7EA0/p1723831259255499

[tnozicka]

For the record, this isn't specific for the helm chart but it's not supported on ScyllaCluster CRD either.
/retitle Allow configuring dedicated volume for ScyllaDB commit log

[scylla-operator-bot]

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 30d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out

/lifecycle stale