Hue 4.9 build shows distutils-related errors with Python 3 and without python3-distutils package

[elukey]

Is the issue already present in https://github.com/cloudera/hue/issues or discussed in the forum https://discourse.gethue.com?

Not that I can see!

Describe the bug:

After building the Vanilla Hue 4.9 from its related branch-4.9.0 I noticed the following error in supervisor.log:

[12/Apr/2021 07:06:39 ] supervisor   ERROR    Exception in supervisor main loop
Traceback (most recent call last):
  File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 360, in main
    for name, supervisee in get_supervisees().items():
  File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 244, in get_supervisees
    return dict((ep.name, ep.load()) for ep in eps)
  File "/usr/lib/hue/desktop/core/src/desktop/supervisor.py", line 244, in <genexpr>
    return dict((ep.name, ep.load()) for ep in eps)
  File "/usr/lib/hue/build/env/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2450, in load
    return self.resolve()
  File "/usr/lib/hue/build/env/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2456, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/hue/desktop/core/src/desktop/kt_renewer.py", line 23, in <module>
    from desktop.conf import KERBEROS as CONF
  File "/usr/lib/hue/desktop/core/src/desktop/conf.py", line 29, in <module>
    from django.db import connection
  File "/usr/lib/hue/build/env/lib/python3.7/site-packages/django/__init__.py", line 1, in <module>
    from django.utils.version import get_version
  File "/usr/lib/hue/build/env/lib/python3.7/site-packages/django/utils/version.py", line 6, in <module>
    from distutils.version import LooseVersion
  File "/usr/lib/hue/build/env/lib/python3.7/distutils/__init__.py", line 25, in <module>
    from distutils import dist, sysconfig
ImportError: cannot import name 'dist' from 'distutils' (/usr/lib/hue/build/env/lib/python3.7/distutils/__init__.py)

This errors clears out on Debian after installing python3-distutils, but I am wondering if anything is missing in the build process since the errors did not come up in previous releases.

Steps to reproduce it?

Just start Hue 4.9 and check the supervisor.log, since the daemon fails to start.

Hue version or source? (e.g. open source 4.5, CDH 5.16, CDP 1.0...). System info (e.g. OS, Browser...).

Open source / vanilla 4.9

[romainr]

Which OS?

[elukey]

Which OS?

Debian 10 :)

[romainr]

Well, for Py3 we added it in the dependencies: https://docs.gethue.com/administrator/installation/dependencies/#ubuntu

[irudayambics]

I am still getting this error, though all dependencies were installed on my Ubuntu 20.04 version.

/home/hue/hue-4.10.0# PREFIX=/home/hue/app make install
"PYTHON_VER is python3.8."
"Python 3 module install via pip"
"SYS_PYTHON is /usr/bin/python3.8."
"ENV_PYTHON is /home/hue/hue-4.10.0/build/env/bin/python3.8."
--- Installing core source structure...
--- Installing Desktop core...
INSTALL_DIR=/home/hue/app/hue make -C desktop install
make[1]: Entering directory '/home/hue/hue-4.10.0/desktop'
"PYTHON_VER is python3.8."
"Python 3 module install via pip"
"SYS_PYTHON is /usr/bin/python3.8."
"ENV_PYTHON is /home/hue/hue-4.10.0/build/env/bin/python3.8."
"----------------> ENV_INSTALL_TARGETS: .recursive-env-install/core .recursive-env-install/libs/aws .recursive-env-install/libs/azure .recursive-env-install/libs/hadoop .recursive-env-install/libs/indexer .recursive-env-install/libs/libanalyze .recursive-env-install/libs/liboauth .recursive-env-install/libs/liboozie .recursive-env-install/libs/librdbms .recursive-env-install/libs/libsaml .recursive-env-install/libs/libsentry .recursive-env-install/libs/libsolr .recursive-env-install/libs/libzookeeper .recursive-env-install/libs/metadata .recursive-env-install/libs/notebook .recursive-env-install/libs/dashboard .recursive-env-install/libs/kafka"
INSTALL_DIR=/home/hue/app/hue/desktop/core
INSTALL_CONF_DIR=/home/hue/app/hue/desktop/conf
make -C core install-bdist
make[2]: Entering directory '/home/hue/hue-4.10.0/desktop/core'
"PYTHON_VER is python3.8."
"Python 3 module install via pip"
"SYS_PYTHON is /usr/bin/python3.8."
"ENV_PYTHON is /home/hue/hue-4.10.0/build/env/bin/python3.8."
--- Building egg for docutils-0.14
Error: The "distutils" standard module, which is required for the
installation of Docutils, could not be found. You may need to
install a package called "python-devel" (or similar) on your
system using your package manager.
make[2]: *** [/home/hue/hue-4.10.0/Makefile.sdk:133: /home/hue/hue-4.10.0/desktop/core/build/docutils-0.14/egg.stamp] Error 1
make[2]: Leaving directory '/home/hue/hue-4.10.0/desktop/core'
make[1]: *** [Makefile:112: .recursive-install-bdist/core] Error 2
make[1]: Leaving directory '/home/hue/hue-4.10.0/desktop'
make: *** [Makefile:132: install-desktop] Error 2

distutils for Python 3.8 has already been installed in my Ubuntu.

$/home/hue# apt-cache search distutils
python-pbr-doc - inject useful and sensible default behaviors into setuptools - doc
python-setuptools-doc - Python Distutils Enhancements (documentation)
python3-distutils - distutils package for Python 3.x
python3-pbr - inject useful and sensible default behaviors into setuptools - Python 3.x
python3-setuptools - Python3 Distutils Enhancements
jcc - code generator producing a Python extension from Java classes
pypy-setuptools - PyPy Distutils Enhancements
python-distutils-extra - enhancements to the Python build system
python-pbr - inject useful and sensible default behaviors into setuptools - Python 2.x
python-pkginfo-doc - Python module to query metadata from packages documentation
python-pytest-runner - Invoke py.test as distutils command with dependency resolution - Python 2.x
python-setuptools - Python Distutils Enhancements
python-sip-dev - Python/C++ bindings generator development files
python3-bdist-nsi - Create NSIS windows installers for Python modules (Python 3)
python3-d2to1 - Python3 support for distutils2-like setup.cfg files as package metadata
python3-distlib - low-level components of python distutils2/packaging
python3-distutils-extra - enhancements to the Python3 build system
python3-fswrap - unified object oriented interface to file system objects (Python 3)
python3-pkginfo - Python 3.x module to query metadata from packages
python3-pytest-runner - Invoke py.test as distutils command with dependency resolution - Python 3.x
python3-sip-dev - Python 3/C++ bindings generator development files
python3-stdeb - Python to Debian source package conversion plugins for distutils
python3-stsci.distutils - Python3 packaging utilities for STScI's packages
python3-versioneer - determine version from repository tag
python3.8-full - Python Interpreter with complete class library (version 3.8)
python3.9-full - Python Interpreter with complete class library (version 3.9)