Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offer improved customization options for user subclasses of UPath #173

Merged
merged 50 commits into from
Feb 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
4f17e39
(wip) customizable fsspec filesystem factory
ap-- Jan 30, 2024
6bb561a
upath.core: fix _default_accessor override error
ap-- Jan 30, 2024
413e011
upath.core: fix _fs_factory call
ap-- Jan 30, 2024
dea4aeb
upath.core: create a patched subclass for _fs_factory
ap-- Jan 30, 2024
780e9d8
upath.core: fix has_custom_fs_factory check
ap-- Jan 30, 2024
a4f2b27
upath._protocol: move to upath protocol parsing
ap-- Feb 1, 2024
c6d600f
tests: allow setting azurite port through envvar
ap-- Feb 1, 2024
fb1cc83
upath._flavour: factor out FSSpecFlavour
ap-- Feb 1, 2024
87c7fbc
upath._protocol: add 'data' protocol matching
ap-- Feb 1, 2024
d1a3af4
upath: allow customization of init args for UPath 3.12+
ap-- Feb 2, 2024
2a3d028
upath._protocol: add allow_unkown kwarg for strip_upath_protocol
ap-- Feb 2, 2024
1eca195
upath.registry: correct class for fsspec file/local
ap-- Feb 4, 2024
6ccddce
upath._flavour: allow unknown protocol
ap-- Feb 4, 2024
9f8598b
upath: use single implementation
ap-- Feb 5, 2024
e9b78d5
upath: make flavour compatible with py3.8+
ap-- Feb 5, 2024
dbb7095
upath.implementations: simplify UPath subclasses
ap-- Feb 5, 2024
48ff891
upath: provide single UPath implementation
ap-- Feb 5, 2024
93b2c09
tests: skip/xfail obsolete tests
ap-- Feb 5, 2024
652f800
tests: memory fs normalization now passes on py38+
ap-- Feb 5, 2024
a2c2098
upath: typo fixes in implementations.local
ap-- Feb 5, 2024
ed3cfb4
upath.implementations.local: fix issues with py312 base
ap-- Feb 5, 2024
21f73dc
upath.implementations.local: fix minor issue in init for WindowsUPath
ap-- Feb 5, 2024
9c7072f
tests: update pathlib3.12 tests to register subclasses
ap-- Feb 5, 2024
f1ea2c8
upath.core: don't promote local paths with storage_options to FilePath
ap-- Feb 5, 2024
2ace188
upath.implementations.webdav: fix webdav for older fsspec versions
ap-- Feb 5, 2024
07703b1
upath.core: port glob fix for older fsspec version
ap-- Feb 5, 2024
2299c82
upath.core: remove inconsistent typing
ap-- Feb 5, 2024
bf5c9fd
upath: fix typing issues
ap-- Feb 5, 2024
7175d72
repo: update pre-commit-config
ap-- Feb 5, 2024
5d09e51
readme: adjust LocalPath -> FilePath
ap-- Feb 5, 2024
28d1f49
upath: deprecate accessors more gracefully
ap-- Feb 5, 2024
6043c50
upath.core: _accessor shim can't use property
ap-- Feb 5, 2024
c893666
upath.implementations.local: fix iterdir for fsspec local paths when …
ap-- Feb 5, 2024
f17782b
upath._compat: fix warnings._deprecated is missing on older pythons
ap-- Feb 6, 2024
e463174
upath.core: try to provide backwards compatibility for custom __new__…
ap-- Feb 6, 2024
60ebee3
upath: align all migration related deprecation warnings
ap-- Feb 6, 2024
4385906
upath.core: add comment explainin _protocol_dispatch
ap-- Feb 6, 2024
3d9a2d2
tests: some subclassing tests in the pathlib3.12 test suite xpass
ap-- Feb 6, 2024
4069227
tests: test the migration compat code with a GitHub implementation
ap-- Feb 6, 2024
7fb8f18
tests: add comment regarding webdav test
ap-- Feb 6, 2024
8e0a389
tests: run github migration test only if GITHUB_TOKEN in env
ap-- Feb 6, 2024
1b29429
tests: make pathlib3.12 tests succeed
ap-- Feb 6, 2024
c41b5c1
upath.core: warn when importing _UriFlavour or PT
ap-- Feb 7, 2024
0e0de47
upath.core: improve comment regarding storage_options combination
ap-- Feb 7, 2024
00aad0a
upath._compat: fix backwards compat for custom accessors
ap-- Feb 7, 2024
aac5fb8
upath: backwards compat support for _drv, _root, and _parts
ap-- Feb 7, 2024
eca6e32
upath: add comment regarding _url
ap-- Feb 7, 2024
b29e836
upath: backward compat support for _path
ap-- Feb 7, 2024
ce2ddd3
upath: provide missing deprecated classmethods on py312
ap-- Feb 7, 2024
31b9877
upath: provide storage_options customization classmethod
ap-- Feb 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ default_language_version:
exclude: ^upath/tests/pathlib/test_pathlib.*\.py|^upath/tests/pathlib/_test_support\.py
repos:
- repo: https://github.com/psf/black
rev: 23.9.1
rev: 24.1.1
hooks:
- id: black
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.5.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
Expand All @@ -25,30 +25,30 @@ repos:
- id: sort-simple-yaml
- id: trailing-whitespace
- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
rev: v2.2.6
hooks:
- id: codespell
additional_dependencies: ["tomli"]
- repo: https://github.com/asottile/pyupgrade
rev: v3.13.0
rev: v3.15.0
hooks:
- id: pyupgrade
args: [--py38-plus]
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
rev: 5.13.2
hooks:
- id: isort
- repo: https://github.com/pycqa/flake8
rev: 6.1.0
rev: 7.0.0
hooks:
- id: flake8
additional_dependencies:
- flake8-bugbear==23.1.20
- flake8-comprehensions==3.10.1
- flake8-bugbear==24.1.17
- flake8-comprehensions==3.14.0
- flake8-debugger==4.1.2
- flake8-string-format==0.3.0
- repo: https://github.com/pycqa/bandit
rev: 1.7.5
rev: 1.7.7
hooks:
- id: bandit
args: [-c, pyproject.toml]
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ For more examples, see the [example notebook here](notebooks/examples.ipynb)

### Currently supported filesystems (and schemes)

- `file:` Local filessystem
- `file:` Local filesystem
- `memory:` Ephemeral filesystem in RAM
- `az:`, `adl:`, `abfs:` and `abfss:` Azure Storage (requires `adlfs` to be installed)
- `http:` and `https:` HTTP(S)-based filesystem
Expand Down Expand Up @@ -99,7 +99,7 @@ flowchart TB
U(UPath)
UP(PosixUPath)
UW(WindowsUPath)
UL(LocalPath)
UL(FilePath)
US3(S3Path)
UH(HttpPath)
UO(...Path)
Expand Down Expand Up @@ -168,13 +168,13 @@ register_implementation(my_protocol, MyPath)

#### Registration via entry points

```toml
```
# pyproject.toml
[project.entry-points."unversal_pathlib.implementations"]
myproto = "my_module.submodule:MyPath"
```

```ini
```
# setup.cfg
[options.entry_points]
universal_pathlib.implementations =
Expand Down
3 changes: 1 addition & 2 deletions noxfile.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Automation using nox."""

import glob
import os

Expand Down Expand Up @@ -46,8 +47,6 @@ def lint(session: nox.Session) -> None:

args = *(session.posargs or ("--show-diff-on-failure",)), "--all-files"
session.run("pre-commit", "run", *args)
# session.run("python", "-m", "mypy")
# session.run("python", "-m", "pylint", *locations)


@nox.session
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ module = "fsspec.*"
ignore_missing_imports = true

[[tool.mypy.overrides]]
module = "webdav4.fsspec.*"
module = "webdav4.*"
ignore_missing_imports = true

[tool.pylint.format]
Expand Down
11 changes: 2 additions & 9 deletions upath/__init__.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,12 @@
"""Pathlib API extended to use fsspec backends."""

import sys

try:
from upath._version import __version__
except ImportError:
__version__ = "not-installed"

if sys.version_info >= (3, 12):
import upath.core312plus as core

sys.modules["upath.core"] = core

else:
import upath.core as core

UPath = core.UPath
from upath.core import UPath

__all__ = ["UPath"]
Loading
Loading