Merge pull request #115 from wazuh/merge-4.10.2-into-master

Merge 4.10.2 into master

.github/workflows/README.md

@@ -0,0 +1,60 @@
+# Installation assistant workflows
+
+This repository includes several GitHub Actions workflows. These workflows are designed to automate the testing process for the installation of the Wazuh Installation Assistant in various environments and to build the different tools and scripts.
+
+## Workflows Overview
+
+1. `Test_installation_assistant`.
+This workflow tests the installation of the Wazuh Installation Assistant in a single-node setup. It triggers on pull requests that modify specific directories or files, and can also be manually dispatched.
+
+2. `Test_installation_assistant_distributed`.
+This workflow is an extension of the Test_installation_assistant workflow, intended for distributed environments. It provisions three instances and simulates a distributed Wazuh deployment across multiple nodes (indexers, managers, and dashboards).
+
+## Triggering the Workflows
+### Automatic Trigger
+The workflows tests are triggered automatically when a pull request (PR) is created or updated, affecting the following paths:
+
+- `cert_tool/`
+- `common_functions/`
+- `config/`
+- `install_functions/`
+- `passwords_tool/`
+- `tests/`
+
+### Manual Trigger
+The test workflows can be triggered manually via the GitHub interface under the "Actions" tab, using the workflow_dispatch event. When triggered manually, several input parameters are required:
+
+- **REPOSITORY**: Defines the repository environment (e.g., staging, pre-release).
+- **AUTOMATION_REFERENCE**: The branch or tag of the `wazuh-automation` repository, used to clone the Allocation module.
+- **SYSTEMS**: A comma-separated list of operating systems to be tested, enclosed in square brackets (e.g., `["CentOS_8", "AmazonLinux_2", "Ubuntu_22", "RHEL8"]`). The available options are: `CentOS_7`, `CentOS_8`, `AmazonLinux_2`, `Ubuntu_16`, `Ubuntu_18`, `Ubuntu_20`, `Ubuntu_22`, `RHEL7`, `RHEL8`.
+- **VERBOSITY**: The verbosity level for Ansible playbook execution, with options `-v`, `-vv`, `-vvv`, and `-vvvv`.
+- **DESTROY**: Boolean value (true or false) indicating whether to destroy the instances after testing. 
+
+## Workflow Structure
+### Jobs
+
+The tests workflows follow a similar structure with the following key jobs:
+
+1. **Checkout Code**: The workflow fetches the latest code from the wazuh-automation and wazuh-installation-assistant repositories.
+
+2. **Set Up Environment**: The operating system is configured based on the selected OS in the SYSTEMS input. The corresponding OS name is stored in the environment variable COMPOSITE_NAME.
+
+3. **Install Ansible**: Ansible is installed for managing the provisioning of instances and running the necessary playbooks.
+
+4. **Provisioning Instances**: The distributed workflow allocates AWS instances using the wazuh-automation repository’s allocation module. It provisions indexers, managers, and dashboards across the instances. The instance inventory is dynamically created and used for later playbook executions.
+
+5. **Ansible Playbooks Execution**: Provision playbooks are executed to prepare the environments for Wazuh components.
+
+6. **Test Execution**: A Python-based testing framework is executed to verify the successful installation and functionality of the Wazuh components on the allocated instances.
+
+7. **Destroy Instances (Optional)**: If the `DESTROY` parameter is set to true, the allocated AWS instances are terminated after the tests. If set to false, the instances and their details are saved as artifacts for later analysis.
+
+### Artifacts
+If instances are not destroyed, the workflow compresses the allocated instances' directory and uploads it as an artifact. Also, the artifacts are compressed with a password. Ask @devel-devops teams for this password. An artifact is uploaded per OS selected. 
+## Notes
+- Instance allocation: The `Test_installation_assistant_distributed` workflow provisions three instances by default. The roles are distributed as follows:
+  - `indexer1`, `indexer2`, `indexer3`: Indexers in the Wazuh cluster.
+  - `master`, `worker1`, `worker2`: Wazuh managers, where `master` is the main manager, and `worker1` and `worker2` are worker nodes.
+  - `dashboard`: Wazuh dashboard.
+
+- Customization: These workflows allow for customization through the various input parameters, making it easy to test different operating systems, verbosity levels, or different versions of the repositories.

.github/workflows/Test_installation_assistant.yml

@@ -1,4 +1,4 @@
-run-name: Test installation assistant - System ${{ inputs.SYSTEM }} - Launched by @${{ github.actor }}
+run-name: Test installation assistant - ${{ github.run_id }} - ${{ inputs.SYSTEMS }} - Launched by @${{ github.actor }}
 name: Test installation assistant
 
 on:
@@ -20,43 +20,193 @@ on:
         options:
           - staging
           - pre-release
+      WAZUH_INSTALLATION_ASSISTANT_REFERENCE:
+        description: 'Branch or tag of the wazuh-installation-assistant repository'
+        required: true
+        default: 'main'
       AUTOMATION_REFERENCE:
-        description: 'wazuh-automation reference'
+        description: 'Branch or tag of the wazuh-automation repository'
+        required: true
+        default: 'master'
+      SYSTEMS:
+        description: 'Operating Systems (list of comma-separated quoted strings enclosed in square brackets)'
         required: true
-        default: 'v5.0.0'
-      SYSTEM:
-        description: 'Operating System'
+        default: '["CentOS_8", "AmazonLinux_2", "Ubuntu_22", "RHEL8"]'
+        type: string
+      VERBOSITY:
+        description: 'Verbosity level on playbooks execution'
         required: true
-        default: 'CentOS 8'
+        default: '-v'
         type: choice
         options:
-          - CentOS 7
-          - CentOS 8
-          - Amazon Linux 2
-          - Ubuntu 16
-          - Ubuntu 18
-          - Ubuntu 20
-          - Ubuntu 22
-          - RHEL7
-          - RHEL8
-      DEBUG:
-        description: 'Debug mode'
-        required: true
-        default: false
-        type: boolean
+          - -v
+          - -vv
+          - -vvv
+          - -vvvv
       DESTROY:
         description: 'Destroy instances after run'
         required: true
         default: true
         type: boolean
 
 env:
-  LABEL: ubuntu-latest
+  COMPOSITE_NAME: "linux-SUBNAME-amd64"
+  SESSION_NAME: "Installation-Assistant-Test"
+  REGION: "us-east-1"
+  TMP_PATH: "/tmp/test"
+  LOGS_PATH: "${{ github.workspace }}/assistant_logs"
+  PKG_REPOSITORY: "${{ inputs.REPOSITORY }}"
+  TEST_NAME: "test_assistant"
+  REPOSITORY_URL: "${{ github.server_url }}/${{ github.repository }}.git"
+  ALLOCATOR_PATH: "/tmp/allocator_instance"
+
+permissions:
+  id-token: write   # This is required for requesting the JWT
+  contents: read    # This is required for actions/checkout
 
 jobs:
-  initialize-environment:
-    runs-on: $LABEL
+  run-test:
+    runs-on: ubuntu-22.04
+    strategy:
+      fail-fast: false    # If a job fails, the rest of jobs will not be canceled
+      matrix:
+        system: ${{ fromJson(inputs.SYSTEMS) }}
 
     steps:
-    - name: Set up Git
-      uses: actions/checkout@v3
+    - name: Checkout code
+      uses: actions/checkout@v4
+      with:
+        ref: ${{ inputs.WAZUH_INSTALLATION_ASSISTANT_REFERENCE }}
+
+    - name: View parameters
+      run: echo "${{ toJson(inputs) }}"
+
+    - name: Set COMPOSITE_NAME variable
+      run: |
+        case "${{ matrix.system }}" in
+          "CentOS_7")
+            SUBNAME="centos-7"
+            ;;
+          "CentOS_8")
+            SUBNAME="centos-8"
+            ;;
+          "AmazonLinux_2")
+            SUBNAME="amazon-2"
+            ;;
+          "Ubuntu_16")
+            SUBNAME="ubuntu-16.04"
+            ;;
+          "Ubuntu_18")
+            SUBNAME="ubuntu-18.04"
+            ;;
+          "Ubuntu_20")
+            SUBNAME="ubuntu-20.04"
+            ;;
+          "Ubuntu_22")
+            SUBNAME="ubuntu-22.04"
+            ;;
+          "RHEL7")
+            SUBNAME="redhat-7"
+            ;;
+          "RHEL8")
+            SUBNAME="redhat-8"
+            ;;
+          *)
+            echo "Invalid SYSTEM selection" >&2
+            exit 1
+            ;;
+        esac
+        COMPOSITE_NAME="${COMPOSITE_NAME/SUBNAME/$SUBNAME}"
+        echo "COMPOSITE_NAME=$COMPOSITE_NAME" >> $GITHUB_ENV
+
+    - name: Install Ansible
+      run: sudo apt-get update && sudo apt install -y python3 && python3 -m pip install --user ansible-core==2.16
+
+    - name: Set up AWS credentials
+      uses: aws-actions/configure-aws-credentials@v4
+      with:
+        role-to-assume: ${{ secrets.AWS_IAM_ROLE }}
+        role-session-name: ${{ env.SESSION_NAME }}
+        aws-region: ${{ env.REGION }}
+
+    - name: Checkout wazuh/wazuh-automation repository
+      uses: actions/checkout@v4
+      with:
+        repository: wazuh/wazuh-automation
+        ref: ${{ inputs.AUTOMATION_REFERENCE }}
+        token: ${{ secrets.GH_CLONE_TOKEN }}
+        path: wazuh-automation
+
+    - name: Install and set allocator requirements
+      run: pip3 install -r wazuh-automation/deployability/deps/requirements.txt
+
+    - name: Allocate instance test and set SSH variables
+      id: allocator_instance
+      run: |
+        python3 wazuh-automation/deployability/modules/allocation/main.py --action create --provider aws --size large --composite-name ${{ env.COMPOSITE_NAME }} --working-dir $ALLOCATOR_PATH \
+          --track-output $ALLOCATOR_PATH/track.yml --inventory-output $ALLOCATOR_PATH/inventory.yml --instance-name gha_${{ github.run_id }}_assistant_test \
+          --label-team devops --label-termination-date 1d
+
+        sed 's/: */=/g' $ALLOCATOR_PATH/inventory.yml > $ALLOCATOR_PATH/inventory_mod.yml
+        sed -i 's/-o StrictHostKeyChecking=no/\"-o StrictHostKeyChecking=no\"/g' $ALLOCATOR_PATH/inventory_mod.yml
+        source $ALLOCATOR_PATH/inventory_mod.yml
+
+        echo "[gha_instance]" > $ALLOCATOR_PATH/inventory
+        echo "$ansible_host ansible_port=$ansible_port ansible_user=$ansible_user ansible_ssh_private_key_file=$ansible_ssh_private_key_file ansible_ssh_common_args='$ansible_ssh_common_args'" >> $ALLOCATOR_PATH/inventory
+
+    - name: Execute provision playbook
+      run: |
+        INSTALL_DEPS=true
+        INSTALL_PYTHON=true
+        INSTALL_PIP_DEPS=true
+
+        ansible-playbook .github/workflows/ansible-playbooks/provision.yml \
+        -i $ALLOCATOR_PATH/inventory \
+        -l all \
+        -e "repository=$REPOSITORY_URL" \
+        -e "reference=${{ github.ref_name }}" \
+        -e "tmp_path=$TMP_PATH" \
+        -e "install_deps=$INSTALL_DEPS" \
+        -e "install_python=$INSTALL_PYTHON" \
+        -e "install_pip_deps=$INSTALL_PIP_DEPS" \
+        "${{ inputs.VERBOSITY }}"
+
+    - name: Execute AIO installation playbook
+      run: |
+        ansible-playbook .github/workflows/ansible-playbooks/aio.yml \
+        -i $ALLOCATOR_PATH/inventory \
+        -l all \
+        -e "tmp_path=$TMP_PATH" \
+        -e "logs_path=$LOGS_PATH" \
+        -e "test_name=$TEST_NAME" \
+        -e "pkg_repository=$PKG_REPOSITORY" \
+        "${{ inputs.VERBOSITY }}"
+
+    - name: Execute Python test playbook
+      run: |
+        TEST_NAME="test_installation_assistant"
+        ansible-playbook .github/workflows/ansible-playbooks/aio_tests.yml \
+        -i $ALLOCATOR_PATH/inventory \
+        -l all \
+        -e "tmp_path=$TMP_PATH" \
+        -e "logs_path=$LOGS_PATH" \
+        -e "test_name=$TEST_NAME" \
+        "${{ inputs.VERBOSITY }}"
+
+    - name: Compress Allocator VM directory
+      id: compress_allocator_files
+      if: always() && steps.allocator_instance.outcome == 'success' && inputs.DESTROY == false
+      run: |
+        zip -P "${{ secrets.ZIP_ARTIFACTS_PASSWORD }}" -r $ALLOCATOR_PATH.zip $ALLOCATOR_PATH
+
+    - name: Upload Allocator VM directory as artifact
+      if: always() && steps.compress_allocator_files.outcome == 'success' && inputs.DESTROY == false
+      uses: actions/upload-artifact@v4
+      with:
+        name: allocator-instance-${{ matrix.system }}
+        path: ${{ env.ALLOCATOR_PATH }}.zip
+
+    - name: Delete allocated VM
+      if: always() && steps.allocator_instance.outcome == 'success' && inputs.DESTROY == true
+      run: python3 wazuh-automation/deployability/modules/allocation/main.py --action delete --track-output $ALLOCATOR_PATH/track.yml
+