Deployment fails from ubuntu 20.04 control node for missing cryptography library

[ukdavo]

Bug description

Deployment to Ubuntu 22.04 LTS is failing at the 'Create private key with password protection' task. The command that I'm running is:

pipenv run ansible-playbook playbooks/acs.yml -i inventory_local.yml -e autogen_unsecure_secrets=true

Target OS

Ubuntu 22.04 LTS

Ansible error

Cannot detect the required Python library cryptography (>= 1.2.3)

Ansible context

Paste the output of the following commands:

ansible --version

pipenv run ansible --version

ansible [core 2.14.4]
  config file = /home/markdav/alfresco-ansible/alfresco-ansible-deployment/ansible.cfg
  configured module search path = ['/home/markdav/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/markdav/.local/share/virtualenvs/alfresco-ansible-deployment-nE5y3qVy/lib/python3.9/site-packages/ansible
  ansible collection location = /home/markdav/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/markdav/.local/share/virtualenvs/alfresco-ansible-deployment-nE5y3qVy/bin/ansible
  python version = 3.9.18 (main, Aug 25 2023, 13:20:04) [GCC 9.4.0] (/home/markdav/.local/share/virtualenvs/alfresco-ansible-deployment-nE5y3qVy/bin/python)
  jinja version = 3.1.2
  libyaml = True

ansible-config dump --only-changed

pipenv run ansible-config dump --only-changed

ANSIBLE_PIPELINING(/home/markdav/alfresco-ansible/alfresco-ansible-deployment/ansible.cfg) = True
CONFIG_FILE() = /home/markdav/alfresco-ansible/alfresco-ansible-deployment/ansible.cfg

ansible-inventory -i your_inventory_file --graph

pipenv run ansible-inventory -i inventory_local.yml --graph

@all:
  |--@ungrouped:
  |--@repository:
  |  |--localhost
  |--@database:
  |  |--@repository:
  |  |  |--localhost
  |--@activemq:
  |  |--@repository:
  |  |  |--localhost
  |--@search:
  |  |--@repository:
  |  |  |--localhost
  |--@search_enterprise:
  |--@elasticsearch:
  |--@identity:
  |--@nginx:
  |  |--@repository:
  |  |  |--localhost
  |--@acc:
  |  |--@repository:
  |  |  |--localhost
  |--@adw:
  |  |--@repository:
  |  |  |--localhost
  |--@transformers:
  |  |--@repository:
  |  |  |--localhost
  |--@syncservice:
  |  |--@repository:
  |  |  |--localhost
  |--@other_repo_clients:
  |--@external_activemq:
  |--@external_elasticsearch:
  |--@external_identity:
  |--@external:
  |  |--@external_activemq:
  |  |--@external_elasticsearch:
  |  |--@external_identity:
  |  |--@other_repo_clients:
  |--@trusted_resource_consumers:
  |  |--@repository:
  |  |  |--localhost
  |  |--@nginx:
  |  |  |--@repository:
  |  |  |  |--localhost
  |  |--@adw:
  |  |  |--@repository:
  |  |  |  |--localhost
  |  |--@other_repo_clients:

[gionn]

Hello, which is the host OS?

Are you sure that while installing the pipenv environment, command completed successfully?

Try running pipenv install --deploy --dev from the root folder and report the output.

The only guess I have for now is that if cryptography has not been installed correctly it's because you are missing some system package required for installing it e.g. build-essential libssl-dev libffi-dev

[ukdavo]

I'm running Ubuntu 20.04 LTS. I think you're right re missing deps. I installed build-essential, libssl-dev & libffi-dev as suggested. Note that I'm running Python 3.9.18. Python 2.x is not installed.

markdav@alf74:~/alfresco-ansible/alfresco-ansible-deployment$ python -V
Python 3.9.18

markdav@alf74:~/alfresco-ansible/alfresco-ansible-deployment$ python3 -V
Python 3.9.18

The Python crypto stuff appears to be present:

markdav@alf74:~/alfresco-ansible/alfresco-ansible-deployment$ pipenv run pip show pyopenssl

Name: pyOpenSSL
Version: 23.2.0
Summary: Python wrapper module around the OpenSSL library
Home-page: https://pyopenssl.org/
Author: The pyOpenSSL developers
Author-email: [email protected]
License: Apache License, Version 2.0
Location: /home/markdav/.local/share/virtualenvs/alfresco-ansible-deployment-nE5y3qVy/lib/python3.9/site-packages
Requires: cryptography
Required-by:

markdav@alf74:~/alfresco-ansible/alfresco-ansible-deployment$ pipenv run pip show cryptography

Name: cryptography
Version: 41.0.4
Summary: cryptography is a package which provides cryptographic recipes and primitives to Python developers.
Home-page:
Author:
Author-email: The Python Cryptographic Authority and individual contributors <[email protected]>
License: Apache-2.0 OR BSD-3-Clause
Location: /home/markdav/.local/share/virtualenvs/alfresco-ansible-deployment-nE5y3qVy/lib/python3.9/site-packages
Requires: cffi
Required-by: ansible-core, pyOpenSSL

markdav@alf74:~/alfresco-ansible/alfresco-ansible-deployment$ pipenv install --deploy --dev
Installing dependencies from Pipfile.lock (a37fec)...
Installing dependencies from Pipfile.lock (a37fec)...

I then retried deployment with the following command but get the same crypto error:

pipenv run ansible-playbook playbooks/acs.yml -i inventory_local.yml --extra-vars "autogen_unsecure_secrets=true ansible_python_interpreter=/usr/bin/python3"

[ukdavo]

I've burned too much time looking at this. I ended up standing up an Ubuntu 22.04 VM instead & was able to deploy to it OK (after remembering to populate known_urls in group_vars/repository.yml).

[gionn]

Please next time provide the full task, not just the error message.

Reproduced at:

TASK [Create private key with password protection] *****************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Cannot detect the required Python library cryptography (>= 1.2.3)"}

PLAY RECAP *********************************************************************************************************************************************************************************************************************************
localhost                  : ok=26   changed=3    unreachable=0    failed=1    skipped=14   rescued=0    ignored=0

(alfresco-ansible-deployment) root@e372626567ec:~/alfresco-ansible-deployment# python --version
Python 3.9.5

reproduction steps:

docker run --rm -it ubuntu:20.04
apt update
apt install python3-pip python3.9-dev git
cd /root/
git clone https://github.com/Alfresco/alfresco-ansible-deployment.git
cd alfresco-ansible-deployment/
pip install --user pipenv
PATH=$PATH:/root/.local/bin
echo $PATH
pipenv install --deploy --python 3.9
pipenv shell
ansible-playbook playbooks/acs.yml -i inventory_local.yml --extra-vars "autogen_unsecure_secrets=true"

[gionn]

apt-get install python3-cryptography or a generic pip3 install cryptography on the control node is fixing the issue, the system python used by ansible requires it

Internally tracked as OPSEXP-2351

[guilloking]

Hello, only install ansible-core, solve the error ubuntu.