dashboard lookup ignores api key

[montaropdf]

SUMMARY

I am trying to collect dashboards using the lookup plugins with an API key.

The plugin generate an exception while switching to the default organization. Normally, this step should be ignored as I specify an api key.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.grafana.grafana_dashboard lookup

ANSIBLE VERSION

ansible [core 2.17.2]
  config file = None
  configured module search path = ['/home/dev/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/dev/.local/lib/python3.12/site-packages/ansible
  ansible collection location = /home/dev/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/dev/.local/bin/ansible
  python version = 3.12.4 (main, Jun  7 2024, 00:00:00) [GCC 13.3.1 20240522 (Red Hat 13.3.1-1)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

# /home/dev/.local/lib/python3.12/site-packages/ansible_collections
Collection        Version
----------------- -------
community.grafana 1.9.1

CONFIGURATION

CONFIG_FILE() = None

OS / ENVIRONMENT

Grafana server version: Grafana v11.1.0 (5b85c4c2fc)

STEPS TO REPRODUCE

In a playbook, targeting localhost, Assign the result of the lookup module to a variable using the set_fact module for example.

The arguments to the lookup module must be 'grafana_dashboard' and a string containing the variables grafana_url and grafana_api_key.

    - name: get all grafana dashboards
      set_fact:
        grafana_dashboards: "{{ lookup('grafana_dashboard', 'grafana_url=http://grafana_server:3000 grafana_api_key=<service_account_token>') }}"

EXPECTED RESULTS

The example above should return all the dashboards in the default organization

ACTUAL RESULTS

ansible-playbook [core 2.17.2]
  config file = None
  configured module search path = ['/home/dev/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/dev/.local/lib/python3.12/site-packages/ansible
  ansible collection location = /home/dev/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/dev/.local/bin/ansible-playbook
  python version = 3.12.4 (main, Jun  7 2024, 00:00:00) [GCC 13.3.1 20240522 (Red Hat 13.3.1-1)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
No config file found; using defaults
setting up inventory plugins
Loading collection ansible.builtin from
host_list declined parsing /home/dev/localhost as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /home/dev/localhost as it did not pass its verify_file() method
auto declined parsing /home/dev/localhost as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /home/dev/localhost as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /home/dev/localhost as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /home/dev/localhost as it did not pass its verify_file() method
[WARNING]: Unable to parse /home/dev/localhost as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
Loading callback plugin default of type stdout, v2.0 from /home/dev/.local/lib/python3.12/site-packages/ansible/plugins/callback/default.py
Attempting to use 'default' callback.
Skipping callback 'default', as we already have a stdout callback.
Attempting to use 'junit' callback.
Attempting to use 'minimal' callback.
Skipping callback 'minimal', as we already have a stdout callback.
Attempting to use 'oneline' callback.
Skipping callback 'oneline', as we already have a stdout callback.
Attempting to use 'tree' callback.

PLAYBOOK: export_dashboards.yaml ********************************************************************************************************************************************************************************
Positional arguments: repos/ostsl-reporting/ansible/export_dashboards.yaml
verbosity: 5
connection: ssh
become_method: sudo
tags: ('all',)
inventory: ('/home/dev/localhost',)
forks: 5
1 plays in repos/ostsl-reporting/ansible/export_dashboards.yaml

PLAY [Export dashboards] ****************************************************************************************************************************************************************************************

TASK [get all grafana dashboards] *******************************************************************************************************************************************************************************
task path: /home/dev/repos/ostsl-reporting/ansible/export_dashboards.yaml:27
redirecting (type: lookup) ansible.builtin.grafana_dashboard to community.grafana.grafana_dashboard
Loading collection community.grafana from /home/dev/.local/lib/python3.12/site-packages/ansible_collections/community/grafana
[DEPRECATION WARNING]: Passing a mangled version of the API key to the grafana_dashboard lookup is no longer necessary and should not be done. This feature will be removed from community.grafana in version
2.0.0. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
exception during Jinja2 execution: Traceback (most recent call last):
  File "/home/dev/.local/lib/python3.12/site-packages/ansible_collections/community/grafana/plugins/lookup/grafana_dashboard.py", line 188, in grafana_list_dashboards
    r = open_url(
        ^^^^^^^^^
  File "/home/dev/.local/lib/python3.12/site-packages/ansible/module_utils/urls.py", line 995, in open_url
    return Request().open(method, url, data=data, headers=headers, use_proxy=use_proxy,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev/.local/lib/python3.12/site-packages/ansible/module_utils/urls.py", line 899, in open
    r = urllib.request.urlopen(request, None, timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 215, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 521, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 630, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 559, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib64/python3.12/urllib/request.py", line 639, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dev/.local/lib/python3.12/site-packages/ansible/template/__init__.py", line 856, in _lookup
    ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev/.local/lib/python3.12/site-packages/ansible_collections/community/grafana/plugins/lookup/grafana_dashboard.py", line 238, in run
    ret = grafana.grafana_list_dashboards()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev/.local/lib/python3.12/site-packages/ansible_collections/community/grafana/plugins/lookup/grafana_dashboard.py", line 196, in grafana_list_dashboards
    raise GrafanaAPIException("Unable to search dashboards : %s" % to_native(e))
ansible_collections.community.grafana.plugins.lookup.grafana_dashboard.GrafanaAPIException: Unable to search dashboards : HTTP Error 401: Unauthorized
fatal: [localhost]: FAILED! => {
    "msg": "An unhandled exception occurred while running the lookup plugin 'grafana_dashboard'. Error was a <class 'ansible_collections.community.grafana.plugins.lookup.grafana_dashboard.GrafanaAPIException'>, original message: Unable to search dashboards : HTTP Error 401: Unauthorized. Unable to search dashboards : HTTP Error 401: Unauthorized"
}

PLAY RECAP ******************************************************************************************************************************************************************************************************
localhost

[Nemental]

Hey @montaropdf,

thank you for reporting this issue!

I'll check the code in the next few days and try to reproduce your problem. However, can you please test your code with the latest version v2.1.0? Thanks :)

[montaropdf]

Hi @Nemental,

I use a repository of packages provided by my company, I hope they will update it soon for testing with the latest version.

Thanks for the quick reply.

[montaropdf]

Hi @Nemental,

I do notice that the last on https://docs.ansible.com/ansible/latest/collections/community/grafana/grafana_dashboard_lookup.html#ansible-collections-community-grafana-grafana-dashboard-lookup

Version 1.9.1 is cited as the latest version available. Is the very latest one yet to be officially published in the ansible collections?

[Nemental]

Hey @montaropdf
unfortunately, the documentation is a bit behind and I don't know of which workflow it's update is part of.
The place of truth for roles and collections is either Ansible Galaxy directly or the release page of the respective repo.

[montaropdf]

@Nemental, the source of the problèm was the ansible python package. it bundles verison 1.9.1 and I was unable to update the collection. I had to remove the package and download the community.grafana collection directly with ansible-galaxy. Now, I have the right version installed and it works perfectly with the example from this issue.

I don't know if you can do something to get the ansible python package updated, but clearly this was the issue.

From my point of view, the case is closed, I will have to ensure the right version is installed.

Thanks for your support.