Windows Python compatibility: Python from Microsoft Store works with Positron; in my experience Python from Python.org does not

[JimCallahanOrlando]

In MS Windows 10 on Intel, among other changes, I updated my version of Python using an .exe downloaded from Python.org and later found that Positron timed out with the Python kernel and complained about not being able to import packages from "ipkernel".

When I uninstalled the Python Loader and Python itself (using "Apps" in "Windows Settings") and then reinstalled Python from Microsoft Store Positron worked again!
Positron notified me it needed to install ipkernel, which I agreed to and now Positron seems to be working fine.

I had also uninstalled Positron 2024.09.0-45 and installed Positron 2024.09.0-77, not certain, but I believe it was the Microsoft Store version of Python that made the difference. I had lost 3 days of productivity (unfortunately Positron is beta software) and my error report on Github appended to a similar complaint about the R interpreter had been marked as "off topic" (so, yes I did do a search for similar discussions and my comment was marked off-topic), so by that point I was in an uninstall-reinstall fury and did not do systematic testing. That's my experience this time, your experience may be different and my experience may be different next time.

From Positron console:

Python 3.12.6 (MicrosoftStore) started.
0.02s - Debugger warning: It seems that frozen modules are being used, which may
0.01s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
Python 3.12.6 (tags/v3.12.6:a4a2d2b, Sep  6 2024, 20:11:23) [MSC v.1940 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.27.0 -- An enhanced Interactive Python. Type '?' for help.
>>> print("Hello World!")
Hello World!
>>>

[JimCallahanOrlando]

PREMATURE OPTIMISM
Although print("Hello World!") and print(2+2) work
as soon as I try to import pandas at the Positron console I get an error:

Python 3.12.6 (MicrosoftStore) started.
Python 3.12.6 (tags/v3.12.6:a4a2d2b, Sep  6 2024, 20:11:23) [MSC v.1940 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.27.0 -- An enhanced Interactive Python. Type '?' for help.
>>> print("Hello World!")
Hello World!
>>> print(2+2)
4
>>> import pandas as pd
ModuleNotFoundError: No module named 'pandas'
Cell In[3], line 1
----> 1 import pandas as pd
>>>

However, if I import a "built-in" module such as "sys" it works, which is fortunate
because then I could print(sys.path) and compare it to other working Python interpreters on my PC.

import sys
print(sys.path)

One of the sources for Python's "sys.path" is an (operating system) environmental variable name "PYTHONPATH".

I checked and "PYTHONPATH" was not set on my system.

Because VS Code's console had a particularly long and thorough "sys.path"
I decided to copy that path to my system's "PYTHONPATH".

Using (and modifying) instructions from TutorialsPoint.
https://www.tutorialspoint.com/how-to-set-python-environment-variable-pythonpath-on-windows

I specifically used (and modified) the instructions for
"Set PYTHONPATH using the Windows Environment Variables dialog"

EXCEPT
From VS Code Console I cut and pasted the output of print(sys.path) to Notepad

because the output is comma-delimited within brackets ['',] and the environmental paths
need quoted paths separated by semi-colons (and no brackets).
So, I

1. In Notepad, I got rid of the brackets at the beginning and end
2. In Notepad, I did a "find and replace" to convert all commas to semi-colons
3. In Notepad, I "Selected All"

Then in the "Windows Environmental Variables dialog" I cut and pasted the resulting string
(as described in TutorialsPoint link above).

Then I checked to make sure the PYTHONPATH was actually set
by opening a CMD line window (aka DOS prompt -- not Powersh...)
and typed

Microsoft Windows [Version 10.0.19045.4894]
(c) Microsoft Corporation. All rights reserved.

C:\Users\JimCa>print %PYTHONPATH%
Unable to initialize device PRN

C:\Users\JimCa>echo %PYTHONPATH%
''; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\DLLs'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\win32'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\win32\\lib'; 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\Pythonwin'

C:\Users\JimCa>

Yes, I forgot you don't "print" environmental variables, you "echo" them,
but the good news is the PYTYONPATH environmental variable was set.

ALMOST WORKED
In Positron, the above string was added to the Python "sys.path".
In other words, print(sys.path) produced this monster!

>>> import sys
>>> print(sys.path)
['c:\\Users\\JimCa\\AppData\\Local\\Programs\\Positron\\resources\\app\\extensions\\positron-python\\python_files\\positron', "D:\\Fdrive\\Quarto\\Population\\''", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\DLLs'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\win32'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\win32\\lib'", "D:\\Fdrive\\Quarto\\Population\\ 'C:\\Users\\JimCa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\Pythonwin'", 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.12_3.12.1776.0_x64__qbz5n2kfra8p0\\python312.zip', 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.12_3.12.1776.0_x64__qbz5n2kfra8p0\\DLLs', 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.12_3.12.1776.0_x64__qbz5n2kfra8p0\\Lib', 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.12_3.12.1776.0_x64__qbz5n2kfra8p0', '', 'C:\\Users\\JimCa\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python312\\site-packages', 'C:\\Users\\JimCa\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python312\\site-packages\\win32', 'C:\\Users\\JimCa\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python312\\site-packages\\win32\\lib', 'C:\\Users\\JimCa\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python312\\site-packages\\Pythonwin', 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.12_3.12.1776.0_x64__qbz5n2kfra8p0\\Lib\\site-packages']
>>>

But, when I tried to import pandas, I was still stuck.

>>> import pandas as pd
ModuleNotFoundError: No module named 'pandas'
Cell In[6], line 1
----> 1 import pandas as pd
>>>

Any suggestions?

[jmcphers]

You may need to install pandas (try pip install pandas at the prompt).

[JimCallahanOrlando]

Pip install worked (slowly), but Positron was still not happy with the path.

>>> pip install pandas
Defaulting to user installation because normal site-packages is not writeable
Collecting pandas
  Downloading pandas-2.2.3-cp312-cp312-win_amd64.whl.metadata (19 kB)
Collecting numpy>=1.26.0 (from pandas)
  Using cached numpy-2.1.1-cp312-cp312-win_amd64.whl.metadata (59 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\jimca\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from pandas) (2.9.0.post0)
Collecting pytz>=2020.1 (from pandas)
  Downloading pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas)
  Downloading tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
Requirement already satisfied: six>=1.5 in c:\users\jimca\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)
Downloading pandas-2.2.3-cp312-cp312-win_amd64.whl (11.5 MB)
   ---------------------------------------- 0.0/11.5 MB ? eta -:--:--
   --- ------------------------------------ 1.0/11.5 MB 12.5 MB/s eta 0:00:01
   -------------------------------- ------- 9.4/11.5 MB 32.5 MB/s eta 0:00:01
   ---------------------------------------- 11.5/11.5 MB 32.7 MB/s eta 0:00:00
Using cached numpy-2.1.1-cp312-cp312-win_amd64.whl (12.6 MB)
Downloading pytz-2024.2-py2.py3-none-any.whl (508 kB)
Downloading tzdata-2024.2-py2.py3-none-any.whl (346 kB)
Installing collected packages: pytz, tzdata, numpy, pandas
Successfully installed numpy-2.1.1 pandas-2.2.3 pytz-2024.2 tzdata-2024.2
Note: you may need to restart the kernel to use updated packages.
  WARNING: The scripts f2py.exe and numpy-config.exe are installed in 'C:\Users\JimCa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
>>>

[jmcphers]

Thanks for reporting this experience. We've had other folks be successful with Python installed from the official sources -- if it's still busted for you in some way, could you follow up here and/or open an issue?

If you use multiple versions of Python, pyenv-win is a great utility that should work well with Positron: https://github.com/pyenv-win/pyenv-win