Multiple zpools per node

[ginn13]

Describe the problem/challenge you have
I need to dynamically provision zfs-backed volumes without redundancy (just single zfs dataset per physical disk). Though it could be simply accomplished with zfs-provisioner on a different nodes, I cannot have more than one volume on a single node, because of StorageClass needs a poolname parameter. So that StorageClass is limited to only one zpool per node.

Trying to set up multiple zpools on a every single node to provision zfs persistent volume on every single drive without redundancy.

Describe the solution you'd like
Support multiple zpools per StorageClass. Allow a list of poolnames in a StorageClass

Environment:

• ZFS-LocalPV version: latest
• Kubernetes version: v1.19.2
• Hardware configuration: 6 baremetal machines with 4 HDD Drives each.
• OS: Centos 7.8.2003

[pawanpraka1]

I cannot have more than one volume on a single node, because of StorageClass needs a poolname parameter.

Why you can't have more that one volume? Poolname in the Storageclass specify that we need to carve out the volume space from that pool. We can have multiple volume provisioed in a single pool via single storage class. Can you elaborate your use case.

[ginn13]

@pawanpraka1, case is: multiple zpools per node. I would like to have ZFSVolume on any of N zpool per single node. However, StorageClass is stick to zpool name, so to the only one zpool on a node.

[nlflint]

I'm pretty sure this functionality already exists. It should be possible for you to apply a second StorageClass and then reference a different zpool in its definition. Then, reference this other storage class in your PV/PVC. I haven't done it before, but I don't see why it wouldn't work.

[Cubixmeister]

@nlflint when using multiple storage classes there will be no load-balancing between different zpool on single machine

[Smithx10]

Also another use case that I ran into.

We deployed many CNs... sometimes our systems netboot with / in ram and have a "data" pool, sometimes our systems boot of the disks from their "system" pool. It would be nice to be able to specify poolname or poolpattern "*/volumes" for the "volumes" storageclass.

[Abhinandan-Purkait]

@ginn13 IIUC, you want to be able to specify a regex pattern for the pool name. This way for example the pattern defined in sc is zpool* and the node has 5 zpools, namely zpool1, zpool2 ... zpool5. Now you can use this same storage class to create volumes on this node on any of the matching zpools?

[Smithx10]

@Abhinandan-Purkait it might be better to match on a custom zpool property set by the administrator on the pool. We can use any pools with matching props. I'd prefer a prop to regex.

[Abhinandan-Purkait]

@Smithx10 Thanks for the suggestion that sounds good to me. Can we have a consensus on the same?

[Abhinandan-Purkait]

Scoping for v4.3, at least have one method of implementation (pattern-matching or zpool property).