Merge pull request #2 from companieshouse/add-iscsi-devices-role

Add iscsi_devices role to collection

roles/informix_management/README.md

@@ -24,15 +24,15 @@ The following role variables are required, however default values may be used wh
 
 | Name                                    | Default    | Description                                            |
 |-----------------------------------------|------------|--------------------------------------------------------|
-| `informix_management_alerts_enabled`    | `no`       | A boolean value representing whether to enable the retrieval of alerts config from Hashicorp Vault. If enabled (i.e. set to `yes`), configuration will be retrieved from Hashicorp Vault using the path specified by the `informix_management_alerts_vault_path` variable. This configuration will be stored in a variable named `informix_management_alerts_config` and can be referenced in Jinja2 templates installed from the path specified by the `informix_management_script_templates_path` variable. |
+| `informix_management_alerts_enabled`    | `false`    | A boolean value representing whether to enable the retrieval of alerts config from Hashicorp Vault. If enabled (i.e. set to `true`), configuration will be retrieved from Hashicorp Vault using the path specified by the `informix_management_alerts_vault_path` variable. This configuration will be stored in a variable named `informix_management_alerts_config` and can be referenced in Jinja2 templates installed from the path specified by the `informix_management_script_templates_path` variable. |
 | `informix_management_alerts_vault_path` |            | The Hashicorp Vault path to read alerts configuration from when `informix_management_alerts_enabled` is true. This information is made accessible to Jinja2 template scripts installed from the `informix_management_script_templates_path` path, using the template variable `informix_management_alerts_config`. |
 | `informix_management_cron_jobs`         |            | See [Informix Cron Jobs][2] for more information.      |
 | `informix_management_informix_group`    | `informix` | The group to be used for ownership of script files, directories, and cron jobs. |
 | `informix_management_informix_user`     | `informix` | The user to be used for ownership of script files and directories. |
 | `informix_management_install_path`      | `/opt/informix/14.10` | The path to the Informix installation directory. |
 | `informix_management_logs_path`         | `/var/log/informix` | The path to a directory which will be created for storing the output of cron job scripts. |
 | `informix_management_script_templates_path` |            | A path to a directory containing one or more [Jinja2](https://jinja.palletsprojects.com/en/2.10.x/) format template script files. These scripts will be installed to `/home/{{ informix_management_informix_user }}/scripts/` and can be executed as cron jobs. See [Informix Cron Jobs][2] for more information. |
-| `informix_management_stats_enabled`     | `no`       | A boolean value representing whether to enable the retrieval of stats config from Hashicorp Vault. If enabled (i.e. set to `yes`), configuration will be retrieved from Hashicorp Vault using the path specified by the `informix_management_stats_vault_path` variable. This configuration will be stored in a variable named `informix_management_stats_config` and can be referenced in Jinja2 templates installed from the path specified by the `informix_management_script_templates_path` variable. |
+| `informix_management_stats_enabled`     | `false`    | A boolean value representing whether to enable the retrieval of stats config from Hashicorp Vault. If enabled (i.e. set to `true`), configuration will be retrieved from Hashicorp Vault using the path specified by the `informix_management_stats_vault_path` variable. This configuration will be stored in a variable named `informix_management_stats_config` and can be referenced in Jinja2 templates installed from the path specified by the `informix_management_script_templates_path` variable. |
 | `informix_management_stats_vault_path`  |            | The Hashicorp Vault path to read stats configuration from when `informix_management_stats_enabled` is true. This information is made accessible to Jinja2 template scripts installed from the `informix_management_script_templates_path` path, using the template variable `informix_management_stats_config`. |
 
 ### Informix Cron Jobs

roles/iscsi_devices/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Crown Copyright (Companies House)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

roles/iscsi_devices/README.md

@@ -0,0 +1,128 @@
+# Ansible Role: iSCSI Devices
+
+An [Ansible Galaxy](https://galaxy.ansible.com/) role for configuring iSCSI devices. Includes scanning for iSCSI targets, configuring multipath and udev rules, creating raw character device nodes, and optionally creating filesystems.
+
+> [!WARNING]
+> There may be a **risk of data loss** when using this role against previously provisioned hosts. For additional safety, this role creates a lock file in `/etc` and will refuse to execute again so long as the lock file exists. See [Reprovisioning Hosts](#reprovisioning-hosts) for more information should this role need to be applied to hosts that contain existing data.
+
+## Table of contents
+
+* [iSCSI Devices][1]
+  * [Raw Character Device Configuration][2]
+  * [Filesystem Configuration][3]
+* [Reprovisioning Hosts][4]
+* [Example Requirements File][5]
+* [Example Playbook][6]
+* [License][7]
+
+[1]: #iscsi-devices
+[2]: #raw-character-device-configuration
+[3]: #filesystem-configuration
+[4]: #reprovisioning-hosts
+[5]: #example-requirements-file
+[6]: #example-playbook
+[7]: #license
+
+## iSCSI Devices
+
+iSCSI devices are configured using a combination of the `iscsi_devices_config` group/host variable and external configuration stored in Hashicorp Vault (internally known by the host fact named `iscsi_devices_vault_config`).
+
+---
+
+The `iscsi_devices_config` variable should be defined as a list of dictionaries, each of which supports the following parameters:
+
+| Name                        | Default | Description                                                                           |
+|-----------------------------|---------|---------------------------------------------------------------------------------------|
+| `alias`                     |         | A unique alias for the storage device.                                                |
+| `multipath`                 |         | A boolean value indicating whether this device uses multipath I/O. Configuration for each multipath device will be added to the file `/etc/multipath.conf`. No configuration will be added for non-multipath devices. |
+| `raw_character_device`      |         | _Optional_. A dictionary specifying raw character device configuration. See [Raw Character Device Configuration][2] for more information. |
+| `filesystem`                |         | _Optional_. A dictionary specifying filesystem configuration. See [Filesystem Configuration][3] for more information. |
+
+> [!NOTE]
+> One of `filesystem` or `raw_character_device` must be provided for each item in the `iscsi_devices_config` list, but not both.
+
+---
+
+The internal host fact `iscsi_devices_vault_config` is set during execution of this role, and its value is retrieved from Hashicorp Vault using the path specified by the mandatory role variable `iscsi_devices_vault_path`. This configuration takes the form of a JSON object with the following attributes:
+
+```json
+{
+  "<ansible-inventory-host>": {
+    "iscsi_device_wwids": {
+      "<alias>": "<wwid>"
+    }
+    "iscsi_initiator_name": "<iscsi-initiator-name>",
+    "iscsi_portal_ips": [
+      "<ip-address>"
+    ]
+  }
+}
+```
+### Raw Character Device Configuration
+
+If defined, the _optional_ `raw_character_device` parameter requires the following:
+
+| Name         | Default | Description                                                                           |
+|--------------|---------|---------------------------------------------------------------------------------------|
+| `group`      |         | The group to be used for ownership of the raw character device node.                  |
+| `owner`      |         | The user to be used for ownership of the raw character device node.                   |
+| `path`       |         | The path to the raw character device node. This should take the form `/dev/raw/raw<N>` where `<N>` is a non-negative integer value (e.g. `/dev/raw/raw1`, `/dev/raw/raw2` and so on). See [raw(8)](https://www.man7.org/linux/man-pages/man8/raw.8.html) for more information. |
+
+> [!NOTE]
+> Each raw character device node is created via a `udev` rule and bound to the WWID of the iSCSI device. The `multipath` setting for each device influences how this `udev` rule behaves:
+>
+> - Multipath devices are assumed to be managed by [Device Mapper](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/index.html) and have a corresponding `/dev/mapper` device node assigned to them. This device node is used in combination with the `DM_UUID` device property value to determine the device's WWID value.
+> - Non-multipath devices are assumed to have kernel names matching `/dev/sd*[!0-9]` (i.e. they are _unpartitioned_) and `scsi_id` is used to determine the device's WWID value.
+>
+> Furthermore, a symbolic link will be created in `/dev` for each device that includes a `raw_character_device` parameter. The name of the symbolic link will match that of the `alias` in the device configuration and will point to the raw character device node. These symbolic links can be used in configuration files to distinguish different raw character devices (e.g. the symbolic link `/dev/scud` in place of `/dev/raw/raw1`).
+
+### Filesystem Configuration
+
+If defined, the _optional_ `filesystem` parameter requires the following:
+
+| Name         | Default | Description                                                                           |
+|--------------|---------|---------------------------------------------------------------------------------------|
+| `group`      |         | The group to be used for ownership of the filesystem mount path.                      |
+| `mode`       |         | The mode to be used for permissions fo the filesystem mount path.                     |
+| `owner`      |         | The user to be used for ownership of the filesystem mount path.                       |
+| `path`       |         | The path where the filesystem will be mounted. This path will be created if it does not already exist. |
+| `type`       |         | The filesystem type to be created. Refer to the Ansible [filesystem module documentation](https://docs.ansible.com/ansible/latest/collections/community/general/filesystem_module.html) for valid options.                                         |
+
+## Reprovisioning Hosts
+
+> [!WARNING]
+> There is a **risk of data loss** if the role is executed against hosts that have previously been provisioned. A lock file is created in `/etc` by this role to provide a basic level of protection against this. The role will refuse to execute again until manual intervention is taken.
+
+To provision a remote host with this role a second time, accepting the potential risk of data loss:
+
+* Stop any active processes on the remote host(s) that may be using the storage device(s)
+* Remove the corresponding lock file from remote host(s) (refer to role variable `iscsi_devices_role_lock_file_path` for the absolute path)
+* Rerun this role against the same host(s)
+
+## Example Requirements File
+
+```yml
+---
+
+collections:
+  - name: companieshouse.middleware
+    version: "1.0.0"
+```
+
+## Example Playbook
+
+```yml
+- name: Provision Informix DB management tooling
+  hosts: all
+  roles:
+    - role: companieshouse.middleware.iscsi_devices
+    - hosts: servers
+      roles:
+        - role: iscsi-devices
+          vars:
+            iscsi_devices_vault_path: /example/path/iscsi
+```
+
+## License
+
+This project is subject to the terms of the [MIT License](LICENSE).

roles/iscsi_devices/defaults/main.yml

@@ -0,0 +1,3 @@
+---
+
+iscsi_devices_role_lock_file_path: /etc/ansible-role-iscsi-devices-provisioned

roles/iscsi_devices/handlers/main.yml

@@ -0,0 +1,15 @@
+---
+
+- name: Restart multipath daemon
+  ansible.builtin.systemd:
+    name: multipathd
+    state: restarted
+    enabled: true
+
+- name: Replay kernel device events
+  ansible.builtin.command:
+    cmd: "{{ item }}"
+  loop:
+    - udevadm control --reload
+    - udevadm trigger -c add
+  changed_when: true

roles/iscsi_devices/meta/main.yml

@@ -0,0 +1,8 @@
+galaxy_info:
+  author: Companies House
+  description: Provision iSCSI devices
+  license: MIT
+  min_ansible_version: 2.15.6
+  galaxy_tags:
+    - iscsi
+  standalone: false

roles/iscsi_devices/tasks/discovery.yml

@@ -0,0 +1,18 @@
+---
+
+- name: Create iSCSI initiator name configuration
+  ansible.builtin.template:
+    src: initiatorname.iscsi.j2
+    dest: /etc/iscsi/initiatorname.iscsi
+    owner: root
+    group: root
+    mode: '0644'
+
+- name: Discover iSCSI targets
+  ansible.builtin.command:
+    cmd: "{{ item }}"
+  loop:
+    - iscsiadm --mode discovery --op update --type sendtargets --portal {{ iscsi_devices_vault_config[ansible_hostname]['iscsi_portal_ips'][0] }}
+    - iscsiadm --mode node -l all
+    - iscsiadm -m session --rescan
+  changed_when: true

roles/iscsi_devices/tasks/filesystem.yml

@@ -0,0 +1,42 @@
+---
+
+- name: "Generate filesystem UUID for iSCSI device '{{ device.alias }}'"
+  ansible.builtin.set_fact:
+    filesystem_uuid: "{{ device.alias | to_uuid }}"
+
+- name: "Create filesystem for iSCSI device '{{ device.alias }}'"
+  community.general.filesystem:
+    fstype: "{{ device.filesystem.type }}"
+    dev: "/dev/disk/by-id/scsi-{{ iscsi_devices_vault_config[ansible_hostname]['iscsi_device_wwids'][device.alias] }}"
+    opts: "-m uuid={{ filesystem_uuid }}"
+    force: true
+  register: filesystem_created
+
+- name: Create filesystem mount point
+  ansible.builtin.file:
+    path: "{{ device.filesystem.path }}"
+    owner: "{{ device.filesystem.owner }}"
+    group: "{{ device.filesystem.group }}"
+    mode: "{{ device.filesystem.mode }}"
+    state: directory
+
+- name: "Mount filesystem for iSCSI device '{{ device.alias }}'" # noqa no-handler
+  ansible.posix.mount:
+    path: "{{ device.filesystem.path }}"
+    src: "UUID={{ filesystem_uuid }}"
+    fstype: "{{ device.filesystem.type }}"
+    opts: "_netdev,x-systemd.requires=iscsi.service"
+    state: mounted
+  when: filesystem_created.changed
+
+- name: Force systemd to reread configs # noqa no-handler
+  ansible.builtin.systemd:
+    daemon_reload: true
+  when: filesystem_created.changed
+
+- name: "Set ownership and permissions for iSCSI mount path '{{ device.filesystem.path }}'"
+  ansible.builtin.file:
+    path: "{{ device.filesystem.path }}"
+    owner: "{{ device.filesystem.owner }}"
+    group: "{{ device.filesystem.group }}"
+    mode: "{{ device.filesystem.mode }}"

roles/iscsi_devices/tasks/main.yml

@@ -0,0 +1,74 @@
+---
+
+- name: Check for presence of role lock file
+  ansible.builtin.stat:
+    path: "{{ iscsi_devices_role_lock_file_path }}"
+  register: lock_result
+
+- name: Assert that lock file does not exist
+  ansible.builtin.fail:
+    msg: >-
+      The host '{{ ansible_hostname }}' was previously provisioned by this role. There is
+      a risk of data loss when running this role against a previously provisioned host.
+      Refer to the README.md file for this role for more information.
+  when: lock_result.stat.exists
+
+- name: Retrieve iSCSI device config from Hashicorp Vault
+  ansible.builtin.set_fact:
+    iscsi_devices_vault_config: "{{ lookup('community.hashi_vault.hashi_vault', iscsi_devices_vault_path) }}"
+
+- name: Check iSCSI device WWIDs are present in Hashicorp Vault
+  ansible.builtin.assert:
+    that:
+      - iscsi_devices_vault_config[ansible_hostname]['iscsi_device_wwids'][device.alias] is defined
+      - iscsi_devices_vault_config[ansible_hostname]['iscsi_device_wwids'][device.alias] | length > 0
+    msg: "Missing Hashicorp Vault iSCSI device WWID for host '{{ ansible_hostname }}' and device '{{ device.alias }}'"
+  loop: "{{ iscsi_devices_config }}"
+  loop_control:
+    loop_var: device
+
+- name: Check iSCSI device configuration contains filesystem or raw character device setting
+  ansible.builtin.assert:
+    that:
+      - device.filesystem is defined or device.raw_character_device is defined
+    msg: "Each element in list iscsi_devices_config must contain a 'filesystem' or 'raw_character_device' atribute"
+  loop: "{{ iscsi_devices_config }}"
+  loop_control:
+    loop_var: device
+
+- name: Check iSCSI initiator name is present in Hashicorp Vault
+  ansible.builtin.assert:
+    that:
+      - iscsi_devices_vault_config[ansible_hostname]['iscsi_initiator_name'] is defined
+      - iscsi_devices_vault_config[ansible_hostname]['iscsi_initiator_name'] | trim | length > 0
+    msg: "Missing Hashicorp Vault iSCSI initiator name for host '{{ ansible_hostname }}'"
+
+- name: Check iSCSI portal IPs are present in Hashicorp Vault
+  ansible.builtin.assert:
+    that:
+      - iscsi_devices_vault_config[ansible_hostname]['iscsi_portal_ips'] is defined
+      - iscsi_devices_vault_config[ansible_hostname]['iscsi_portal_ips'] | length > 0
+    msg: "Missing Hashicorp Vault iSCSI portal IPs for host '{{ ansible_hostname }}'"
+
+- name: Discover iSCSI devices
+  ansible.builtin.import_tasks: discovery.yml
+
+- name: Provision multipath configuration
+  ansible.builtin.import_tasks: multipath.yml
+
+- name: Provision udev rules
+  ansible.builtin.import_tasks: udev.yml
+
+- name: Provision filesystem(s)
+  ansible.builtin.include_tasks: filesystem.yml
+  loop: "{{ iscsi_devices_config | selectattr('filesystem', 'defined') | list }}"
+  loop_control:
+    loop_var: device
+
+- name: Create role lock file
+  ansible.builtin.copy:
+    content: "{{ ansible_date_time.iso8601 }} : Provisioned by 'ansible-role-iscsi-devices' role\n"
+    dest: "{{ iscsi_devices_role_lock_file_path }}"
+    owner: root
+    group: root
+    mode: '0444'

roles/iscsi_devices/tasks/multipath.yml

@@ -0,0 +1,17 @@
+---
+
+- name: Create multipath daemon configuration
+  ansible.builtin.template:
+    src: multipath.conf.j2
+    dest: /etc/multipath.conf
+    owner: root
+    group: root
+    mode: '0600'
+    trim_blocks: true
+  vars:
+    multipath_devices: "{{ iscsi_devices_config | selectattr('multipath') | list }}"
+  notify:
+    - Restart multipath daemon
+
+- name: Flush handlers
+  ansible.builtin.meta: flush_handlers