pyspedas 2.0.0 migration guide needed

[jameswilburlewis]

There are some changes coming in pyspedas 2.0 that users will need to know about and adapt to. Most of the necessary changes will work in recent pyspedas 1.X.Y releases, so users can start adopting these practices now. We should add a pyspedas 2.0 migration guide to our readthedocs page, and update our notebooks and tutorials to follow the new best practices.

1. Mission specific load and analysis routine moved to projects directory

The themis, mms, erg and most other project directories have been moved into a 'projects' directory. We have an import-time kludge implemented so that calls like 'pyspedas.themis.fgm()' or 'from pyspedas.themis import fgm' still work, but at least PyCharm's static analyzer doesn't see the runtime state of the pyspedas namespace, so it flags some of the constructs as errors now, even though they actually work just fine at runtime.

If we can't figure out how to make this work cleanly, we might want to deprecate pyspedas.themis, pyspedas.mms, etc in favor of 'pyspedas.projects.themis', 'pyspedas.projects.mms', etc

2. More directory structure changes: the cotrans module has moved to cotrans_tools, cdagui to cdagui_tools, hapi to hapi_tools, to avoid clashes between the cotrans(), hapi() and cdagui() functions and their respective module names.

Similar changes will probably be implemented for MMS subdirectories which conflict with function names (e.g. fgm(), feeps(), etc)

Users who are importing from these directories will need to use the new names (or preferably import directly from the pyspedas top level namespace or mms namespace, rather than drilling down to specific file names).

3. pytplot to be absorbed into pyspedas

It will no longer be necessary or recommended to "import pytplot". All the tplot routines are already imported into the pyspedas namespace, so "from pyspedas import tplot, get_data, store_data" can replace the equivalent imports from pytplot.

[Beforerr]

Since version 2.0 may introduce breaking changes, I propose we switch to using a local container, such as a dict, to store all tplot variables, instead of the current global container pytplot.data_quants. For backward compatibility, we could introduce a keyword argument, like global, set to True by default in load routines. This would ensure variables are copied from the local dict to the global one as needed. Using a local structure offers significant benefits for data processing and parallelization, aligning more closely with Python best practices. Since most routines rely on store_data as a core function, the necessary code adjustments should be minimal.

[jameswilburlewis]

@Beforerr I've been thinking about introducing some sort of "tplot workspace" object that would serve as a container for tplot variables. There are times when we would want to load some support data for an intermediate calculation, but without overwriting existing variables -- it can often be done by passing suffixes around, but that gets messy and error-prone. It probably won't be part of pyspedas 2.0.0 yet.

[jameswilburlewis]

Perhaps we should also pull some pyspedas components out into optional addons for pyspedas 2.0.0:

secs mission: depends on basemap, which is an ongoing source of problems due to their practice of pinning upper version limits on their package requirements. See: #1057

geopack: Dependency on the geopack package is currently blocking us from getting on conda-forge (though this could probably be resolved with a bit of work, since it's a pure python package without any complicated dependencies).

So "pip install pyspedas" might omit secs and geopack. "pip install pyspedas[secs]" or "pip install pyspedas[all]" could be used to install optional dependencies if your environment is supported by those packages.

This might imply making pyspedas a namespace package?

[Beforerr]

I believe it would be beneficial to make certain features optional, as not all users require all the functionalities provided by pyspedas. This approach works particularly well with the lazy loading mechanism, no loading cost for something you never use —please refer to the pull request #1052. Additionally, it sounds fairer to make optional any components that introduce additional dependencies (like viresclient, making viresclient as optional could help remove table dep #792)