Skip to content
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.

Bump psycopg2-binary dependency #82

Open
astrojuanlu opened this issue May 3, 2022 · 2 comments
Open

Bump psycopg2-binary dependency #82

astrojuanlu opened this issue May 3, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@astrojuanlu
Copy link

Is your feature request related to a problem? Please describe.
The currently pinned version of psycopg2-binary is a bit old (2.8.5, released Apr 6, 2020, two years ago) and does not ship precompiled wheels for Python 3.9 onwards, making it a bit annoying on newer Python versions.

Describe the solution you'd like
Increase the pinned version of psycopg2-binary (for instance, merging #65).

Describe alternatives you've considered

  • Avoid strict pinnings. But I anticipate that, since Singer taps and targets are meant to be treated as applications rather than libraries, developers probably want to keep tight control on the dependecies.
  • Don't do anything. It's not the end of the world, either I can create a Python 3.8 environment (latest version with wheels) or just install libpq-dev and a compiler. But it's still mildly annoying.
@astrojuanlu astrojuanlu added the enhancement New feature or request label May 3, 2022
@alxthm
Copy link

alxthm commented May 23, 2022

For reference, here is the error I get when trying to install this target with python 3.9.12:

Installing 4 plugins...
Installing file bundle 'dbt'...
Installing transformer 'dbt'...
Installed file bundle 'dbt'
Run `meltano upgrade files` to update your project's 'dbt' files.
Installing loader 'target-postgres'...
Installed transformer 'dbt'
Installing extractor 'tap-csv'...
Loader 'target-postgres' could not be installed: failed to install plugin 'target-postgres'.
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.9
      creating build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extras.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/tz.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_json.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extensions.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/compat.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errorcodes.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/pool.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/__init__.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errors.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/sql.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_range.py -> build/lib.linux-x86_64-3.9/psycopg2
      running build_ext
      building 'psycopg2._psycopg' extension
      creating build/temp.linux-x86_64-3.9
      creating build/temp.linux-x86_64-3.9/psycopg
      gcc -pthread -B /root/miniconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/include -I/root/miniconda3/include -fPIC -O2 -isystem /root/miniconda3/include -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=130005 -DHAVE_LO64=1 -I/root/meltano_example_implementations/meltano_projects/singer_dbt_jaffle/.meltano/loaders/target-postgres/venv/include -I/root/miniconda3/include/python3.9 -I. -I/usr/include/postgresql -I/usr/include/postgresql/13/server -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-3.9/psycopg/adapter_asis.o -Wdeclaration-after-statement
      In file included from psycopg/adapter_asis.c:28:
      ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
         36 | #include <libpq-fe.h>
            |          ^~~~~~~~~~~~
      compilation terminated.

      It appears you are missing some prerequisite to build the package from source.

      You may install a binary package by installing 'psycopg2-binary' from PyPI.
      If you want to install psycopg2 from source, please install the packages
      required for the build and try again.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for psycopg2-binary
  error: subprocess-exited-with-error

  × Running setup.py install for psycopg2-binary did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      running install
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.9
      creating build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extras.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/tz.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_json.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extensions.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/compat.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errorcodes.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/pool.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/__init__.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errors.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/sql.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_range.py -> build/lib.linux-x86_64-3.9/psycopg2
      running build_ext
      building 'psycopg2._psycopg' extension
      creating build/temp.linux-x86_64-3.9
      creating build/temp.linux-x86_64-3.9/psycopg
      gcc -pthread -B /root/miniconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/include -I/root/miniconda3/include -fPIC -O2 -isystem /root/miniconda3/include -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=130005 -DHAVE_LO64=1 -I/root/meltano_example_implementations/meltano_projects/singer_dbt_jaffle/.meltano/loaders/target-postgres/venv/include -I/root/miniconda3/include/python3.9 -I. -I/usr/include/postgresql -I/usr/include/postgresql/13/server -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-3.9/psycopg/adapter_asis.o -Wdeclaration-after-statement
      In file included from psycopg/adapter_asis.c:28:
      ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
         36 | #include <libpq-fe.h>
            |          ^~~~~~~~~~~~
      compilation terminated.

      It appears you are missing some prerequisite to build the package from source.

      You may install a binary package by installing 'psycopg2-binary' from PyPI.
      If you want to install psycopg2 from source, please install the packages
      required for the build and try again.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> psycopg2-binary

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Installed extractor 'tap-csv'
Installed 3/4 plugins
Failed to install plugin(s)

Fixed by installing libpq-dev

@cowiekw
Copy link

cowiekw commented Nov 22, 2022

For reference, here is the error I get when trying to install this target with python 3.9.12:

Installing 4 plugins...
Installing file bundle 'dbt'...
Installing transformer 'dbt'...
Installed file bundle 'dbt'
Run `meltano upgrade files` to update your project's 'dbt' files.
Installing loader 'target-postgres'...
Installed transformer 'dbt'
Installing extractor 'tap-csv'...
Loader 'target-postgres' could not be installed: failed to install plugin 'target-postgres'.
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.9
      creating build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extras.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/tz.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_json.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extensions.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/compat.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errorcodes.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/pool.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/__init__.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errors.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/sql.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_range.py -> build/lib.linux-x86_64-3.9/psycopg2
      running build_ext
      building 'psycopg2._psycopg' extension
      creating build/temp.linux-x86_64-3.9
      creating build/temp.linux-x86_64-3.9/psycopg
      gcc -pthread -B /root/miniconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/include -I/root/miniconda3/include -fPIC -O2 -isystem /root/miniconda3/include -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=130005 -DHAVE_LO64=1 -I/root/meltano_example_implementations/meltano_projects/singer_dbt_jaffle/.meltano/loaders/target-postgres/venv/include -I/root/miniconda3/include/python3.9 -I. -I/usr/include/postgresql -I/usr/include/postgresql/13/server -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-3.9/psycopg/adapter_asis.o -Wdeclaration-after-statement
      In file included from psycopg/adapter_asis.c:28:
      ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
         36 | #include <libpq-fe.h>
            |          ^~~~~~~~~~~~
      compilation terminated.

      It appears you are missing some prerequisite to build the package from source.

      You may install a binary package by installing 'psycopg2-binary' from PyPI.
      If you want to install psycopg2 from source, please install the packages
      required for the build and try again.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for psycopg2-binary
  error: subprocess-exited-with-error

  × Running setup.py install for psycopg2-binary did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      running install
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.9
      creating build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extras.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/tz.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_json.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/extensions.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/compat.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errorcodes.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/pool.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/__init__.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/errors.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/sql.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.9/psycopg2
      copying lib/_range.py -> build/lib.linux-x86_64-3.9/psycopg2
      running build_ext
      building 'psycopg2._psycopg' extension
      creating build/temp.linux-x86_64-3.9
      creating build/temp.linux-x86_64-3.9/psycopg
      gcc -pthread -B /root/miniconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /root/miniconda3/include -I/root/miniconda3/include -fPIC -O2 -isystem /root/miniconda3/include -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=130005 -DHAVE_LO64=1 -I/root/meltano_example_implementations/meltano_projects/singer_dbt_jaffle/.meltano/loaders/target-postgres/venv/include -I/root/miniconda3/include/python3.9 -I. -I/usr/include/postgresql -I/usr/include/postgresql/13/server -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-3.9/psycopg/adapter_asis.o -Wdeclaration-after-statement
      In file included from psycopg/adapter_asis.c:28:
      ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
         36 | #include <libpq-fe.h>
            |          ^~~~~~~~~~~~
      compilation terminated.

      It appears you are missing some prerequisite to build the package from source.

      You may install a binary package by installing 'psycopg2-binary' from PyPI.
      If you want to install psycopg2 from source, please install the packages
      required for the build and try again.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> psycopg2-binary

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Installed extractor 'tap-csv'
Installed 3/4 plugins
Failed to install plugin(s)

Fixed by installing libpq-dev

I have the same error while installing this target in meltano. How did you install libpq-dev ?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@astrojuanlu @alxthm @cowiekw