[Bug]: netbox.nb_lookup fails with self-signed certificates.

[joubbi]

ISSUE TYPE

• Bug Report

SOFTWARE VERSIONS

Ansible:

ansible [core 2.12.1]
python version 3.9.9 [GCC 11.2.0]
jinja version = 2.11.3
libyaml = True

NetBox:

v3.1.5

Collection:

3.5.1

SUMMARY

I have a server running Netbox with self-signed certificates.
Using netbox.nb_lookup against it fails with SSL: CERTIFICATE_VERIFY_FAILED even though I have specified validate_certs='no'.

The plugin netbox.nb_inventory works fine against the same Netbox server as long as I specify validate_certs: False.
The same playbook (with modified token and api_endpoint) works against another install of Netbox with a properly signed certificate.

STEPS TO REPRODUCE

- name: "Test Netbox modules"
  connection: local
  hosts: localhost
  gather_facts: False

  tasks:
    - name: Obtain list of devices from Netbox
      debug:
        msg: >
          "Device {{ item.value.display_name }} (ID: {{ item.key }}) was
           manufactured by {{ item.value.device_type.manufacturer.name }}"
      loop: "{{ query('netbox.netbox.nb_lookup', 'devices',
                      validate_certs='no',
                      api_endpoint='https://10.0.0.1/',
                      token='123') }}"

EXPECTED RESULTS

A list of devices should be printed out.

ACTUAL RESULTS

TASK [Obtain list of devices from Netbox] **********************************************************************************************************************************

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'netbox.netbox.nb_lookup'. Error was a <class 'requests.exceptions.SSLError'>, original message: HTTPSConnectionPool(host='10.0.0.1', port=443): Max retries exceeded with url: /api/dcim/devices/?limit=0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)'))). HTTPSConnectionPool(host='10.0.0.1', port=443): Max retries exceeded with url: /api/dcim/devices/?limit=0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)')))"}

[rodvand]

Try using validate_certs=False in the loookup. I think it's a bug in the documentation which we haven't managed to fix because ansible-doc keep changing it back to yes/no.

[joubbi]

I tried with validate_certs=False. Same result.

[rodvand]

In an attempt to reproduce:
Ansible version:

ansible [core 2.12.1]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/rodvand/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/rodvand/venv/lib/python3.8/site-packages/ansible
  ansible collection location = /home/rodvand/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/rodvand/venv/bin/ansible
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
  jinja version = 3.0.3
  libyaml = True

Playbook:

- name: "Test Netbox modules"
  connection: local
  hosts: localhost
  gather_facts: False

  tasks:
    - name: Obtain list of devices from Netbox
      debug:
        msg: "{{ item }}"
      loop: "{{ query('netbox.netbox.nb_lookup', 'devices',
                      validate_certs=False,
                      api_endpoint='https://*****/',
                      token='****') }}"

Output:

(venv) rodvand@fyb-dev:~$ ansible-playbook test-play.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Test Netbox modules] ********************************************************************************************************************

TASK [Obtain list of devices from Netbox] *****************************************************************************************************
/home/rodvand/venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'nb29.rodvand.net'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
ok: [localhost] => (item={'key': 1, 'value': {'id': 1, 'url': 'https://***/api/dcim/devices/1/', 'name': 'First Firewall', 'display_name': 'First Firewall', 'device_type': {'id': 1, 'url': 'https://***/api/dcim/device-types/1/', 'manufacturer': {'id': 1, 'url': 'https://***/api/dcim/manufacturers/1/', 'name': 'Check Point', 'slug': 'check-point'}, 'model': 'Firewall 1', 'slug': 'firewall-1', 'display_name': 'Check Point Firewall 1'}, 'device_role': {'id': 1, 'url': 'https://***/api/dcim/device-roles/1/', 'name': 'Role ONE', 'slug': 'role-one'}, 'tenant': None, 'platform': None, 'serial': '123456789', 'asset_tag': None, 'site': {'id': 1, 'url': 'https://***/api/dcim/sites/1/', 'name': 'Site 1', 'slug': 'site-1'}, 'rack': None, 'position': None, 'face': None, 'parent_device': None, 'status': {'value': 'active', 'label': 'Active'}, 'primary_ip': None, 'primary_ip4': None, 'primary_ip6': None, 'cluster': None, 'virtual_chassis': None, 'vc_position': None, 'vc_priority': None, 'comments': '', 'local_context_data': None, 'tags': [], 'custom_fields': {}, 'config_context': {}, 'created': '2022-01-25', 'last_updated': '2022-01-25T20:43:44.009238+01:00'}}) => {
    "msg": "First Firewall"
}

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

If I change validate_certs to True:

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'netbox.netbox.nb_lookup'. Error was a <class 'requests.exceptions.SSLError'>, original message: HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: /api/dcim/devices/?limit=0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1131)'))). HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: /api/dcim/devices/?limit=0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1131)')))"}

My python packages:

(venv) rodvand@fyb-dev:~$ pip freeze
ansible==5.2.0
ansible-core==2.12.1
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.10
cryptography==36.0.1
idna==3.3
Jinja2==3.0.3
MarkupSafe==2.0.1
packaging==21.3
pycparser==2.21
pynetbox==6.5.0
pyparsing==3.0.7
PyYAML==6.0
requests==2.27.1
resolvelib==0.5.4
six==1.16.0
urllib3==1.26.8

And ansible collection:

(venv) rodvand@fyb-dev:~$ ansible-galaxy collection list | grep netbox
netbox.netbox 3.5.1

So I don't think there's anything wrong with the ansible collection. Have you tried doing a straight up pynetbox connection and getting it to work?

[joubbi]

Sorry for a late reply.
We have replaced the self-signed certificate with a signed certificate.
I copied your playbook and ran it. I get this:

PLAY [Test Netbox modules] *************************************************************************************************************************************************

 

TASK [Obtain list of devices from Netbox] **********************************************************************************************************************************

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'netbox.netbox.nb_lookup'. Error was a <class 'requests.exceptions.SSLError'>, original message: HTTPSConnectionPool(host='[example.com](http://example.com/)', port=443): Max retries exceeded with url: /api/dcim/devices/?limit=0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))). HTTPSConnectionPool(host='[example.com](http://example.com/)', port=443): Max retries exceeded with url: /api/dcim/devices/?limit=0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))"}

 

PLAY RECAP *****************************************************************************************************************************************************************

localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Any ideas on how to troubleshoot this further?
Note: nb_inventory works fine against the same host using the same URL and token.

[joubbi]

I found the problem.
The server certificate is signed by an intermediate CA, which in turn is signed by another intermediate CA, which is signed by a root CA.
One of the intermediate certificates is not in the certificate bundle and it was not presented by the server.
I added the whole chain of certificates to nginx on the netbox server. Now it works. Problem solved.

Still I think it is a bug that validate_certs=False is not letting me connect if the intermediate certificate is missing.
The other module nb_inventoryhandled the situation as expected.