Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs improvements #187

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/contributing/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Before submitting we recommend checking a few things

Do's and Dont's
----------------
- Do keep your PR scope small. We would rather review many small PRs with seperate features than one giant one.
- Do keep your PR scope small. We would rather review many small PRs with separate features than one giant one.
- Make draft PRs.

Project structure
Expand All @@ -45,6 +45,6 @@ Nextcore is currently split into 3 main modules
- nextcore.http
- nextcore.gateway

Common is for common utilies that needs to be shared between the other modules.
Common is for common utilities that needs to be shared between the other modules.
HTTP is for the REST API.
Gateway is for the WebSocket gateway.
4 changes: 2 additions & 2 deletions docs/events.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This is a document showing you the arguments from the different instances of :cl

Raw Dispatcher
--------------
Can be found on :attr:`ShardManadger.raw_dispatcher <gateway.ShardManager.raw_dispatcher>` and :attr:`Shard.raw_dispatcher <gateway.Shard.raw_dispatcher>`.
Can be found on :attr:`ShardManager.raw_dispatcher <gateway.ShardManager.raw_dispatcher>` and :attr:`Shard.raw_dispatcher <gateway.Shard.raw_dispatcher>`.
These are the raw dispatchers that just relay raw events from the discord websocket (the gateway).

The event name here is the gateway `opcode <https://discord.dev/docs/topics/gateway#gateway-opcodes>`__.
Expand All @@ -22,7 +22,7 @@ The event name here is the gateway `opcode <https://discord.dev/docs/topics/gate

Event Dispatcher
----------------
Can be found on :attr:`ShardManadger.event_dispatcher <gateway.ShardManager.event_dispatcher>` and :attr:`Shard.event_dispatcher <gateway.Shard.event_dispatcher>`.
Can be found on :attr:`ShardManager.event_dispatcher <gateway.ShardManager.event_dispatcher>` and :attr:`Shard.event_dispatcher <gateway.Shard.event_dispatcher>`.
These dispatchers dispatch the data inside the ``d`` key of a :attr:`GatewayOpcode.DISPATCH` event.

The event name is the Dispatch `event name <https://discord.dev/topics/gateway#commands-and-events-gateway-events>`__.
Expand Down
4 changes: 2 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,5 @@ The documentation will now split into different pages depending on what function

Helping out
=============
We would appriciate your help writing nextcore and related libraries.
See :ref:`contributing` for more info
We would appreciate your help in developing nextcore and its related libraries.
Please see :ref:`contributing` for more info
32 changes: 32 additions & 0 deletions docs/serve_dev.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# The MIT License (MIT)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# Copyright (c) 2021-present nextcore developers
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.

from livereload import Server, shell

if __name__ == '__main__':
shell("make.bat html") # initial make
server = Server()
server.watch('*.rst', shell('make.bat html'), delay=1)
server.watch('*.md', shell('make.bat html'), delay=1)
server.watch('*.py', shell('make.bat html'), delay=1)
server.watch('_static/*', shell('make.bat html'), delay=1)
server.watch('contributing/*', shell('make.bat html'), delay=1)
server.serve(root='_build/html')
4 changes: 2 additions & 2 deletions nextcore/common/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
if TYPE_CHECKING:
from typing import Final

__all__: Final[tuple[str, ...]] = ()
__all__: Final[tuple[str, ...]] = ("RateLimitedError",)


class RateLimitedError(Exception):
"""A error for when a :class:`~nextcore.common.TimesPer` is rate limited and ``wait`` was :data:`False`"""
"""An error for when a :class:`~nextcore.common.TimesPer` is rate limited and ``wait`` was :data:`False`"""
4 changes: 2 additions & 2 deletions nextcore/http/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@

"""Do requests to Discord over the HTTP API.

This module includes a HTTP client that handles rate limits for you,
and gives you convinient methods around the API.
This module includes an HTTP client that handles rate limits for you,
and gives you convenient methods around the API.
"""

from .authentication import *
Expand Down
22 changes: 11 additions & 11 deletions nextcore/http/bucket.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ class Bucket:

def __init__(self, metadata: BucketMetadata):
self.metadata: BucketMetadata = metadata
self._remaining: int | None = None # None signifies unlimited or not used yet (due to a optimization)
self._remaining: int | None = None # None signifies unlimited or not used yet (due to an optimization)
self._pending: PriorityQueue[RequestSession] = PriorityQueue()
self._reserved: list[RequestSession] = []
self._resetting: bool = False
Expand All @@ -66,7 +66,7 @@ def __init__(self, metadata: BucketMetadata):

@asynccontextmanager
async def acquire(self, *, priority: int = 0, wait: bool = True) -> AsyncIterator[None]:
"""Use a spot in the rate limit.
"""Uses a spot in the rate limit.

Parameters
----------
Expand Down Expand Up @@ -105,9 +105,9 @@ async def acquire(self, *, priority: int = 0, wait: bool = True) -> AsyncIterato
raise RateLimitedError()
self._pending.put_nowait(
session
) # This can't raise a exception as pending is always infinite unless someone else modified it
) # This can't raise an exception as pending is always infinite unless someone else modified it
await session.pending_future # Wait for a spot in the rate limit.
# This will automatically be removed by the waker.
# This will automatically be removed by the waker. TODO: maybe find out what this is and describe it.

self._reserved.append(session)
try:
Expand All @@ -122,10 +122,10 @@ async def acquire(self, *, priority: int = 0, wait: bool = True) -> AsyncIterato
self._reserved.remove(session)
return

# We have no info on rate limits, so we have to do a "blind" request to find out what the rate limits is.
# We will only do one "blind" request at a time per bucket though in case the rate limit is small.
# This could be tweaked to use more on routes with higher rate limits, however this would require hard coding which is not a thing I want
# for nextcore.
# We have no info on rate limits, so we have to do a "blind" request to find out what the rate limits is. We
# will only do one "blind" request at a time per bucket though in case the rate limit is small. This could be
# tweaked to use more on routes with higher rate limits, however this would require complex code which is not
# a thing I want for nextcore. # TODO: maybe something to mention in the contributor docs?
session = RequestSession(priority=priority)

if self._can_do_blind_request.is_set():
Expand Down Expand Up @@ -191,7 +191,7 @@ async def update(

def _reset_callback(self) -> None:
self._resetting = False # Allow future resets
self._remaining = None # It should use metadata's limit as a starting point.
self._remaining = None # It should use the metadata limit as a starting point.

# Reset up to the limit
self._release_pending(self.metadata.limit)
Expand All @@ -203,9 +203,9 @@ def _release_pending(self, max_count: int | None = None):
max_count = min(max_count, self._pending.qsize())

for _ in range(max_count):
session = self._pending.get_nowait() # This can't raise a exception due to the guard clause.
session = self._pending.get_nowait() # This can't raise an exception due to the guard clause.

# Mark it as completed in the queue to avoid a infinitly overflowing int
# Mark it as completed in the queue to avoid an infinitely overflowing int
self._pending.task_done()

session.pending_future.set_result(None)
Expand Down
4 changes: 2 additions & 2 deletions nextcore/http/bucket_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class BucketMetadata:
The maximum number of requests that can be made in the given time period.
unlimited:
Whether the bucket has an unlimited number of requests. If this is :class:`True`,
limit has to be None.
limit must be None.

Attributes
----------
Expand All @@ -50,7 +50,7 @@ class BucketMetadata:

This will also be :data:`None` if no limit has been fetched yet.
unlimited:
Wheter the bucket has no rate limiting enabled.
Whether the bucket has an unlimited number of requests.
"""

__slots__ = ("limit", "unlimited")
Expand Down
25 changes: 14 additions & 11 deletions nextcore/http/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ class RateLimitingFailedError(Exception):
"""When rate limiting has failed more than :attr:`HTTPClient.max_retries` times

.. hint::
This can be due to a un-syncronized clock.
This can be due to an un-synchronised clock.

You can change :attr:`HTTPClient.trust_local_time` to :data:`False` to disable using your local clock,
or you could sync your clock.
Expand Down Expand Up @@ -100,14 +100,14 @@ class RateLimitingFailedError(Exception):
Parameters
----------
max_retries:
How many retries the request used that failed.
How many retries were allowed for the request.
response:
The response to the last request that failed.

Attributes
----------
max_retries:
How many retries the request used that failed.
How many retries were allowed for the request.
response:
The response to the last request that failed.
"""
Expand Down Expand Up @@ -139,7 +139,7 @@ class HTTPRequestStatusError(Exception):
message:
The error message.
error:
The error json from the body.
The error in json from the body.
"""

def __init__(self, error: HTTPErrorResponseData, response: ClientResponse) -> None:
Expand All @@ -153,29 +153,32 @@ def __init__(self, error: HTTPErrorResponseData, response: ClientResponse) -> No
super().__init__(f"({self.error_code}) {self.message}")


# TODO: Can the docstrings be improved here?
class BadRequestError(HTTPRequestStatusError):
"""A 400 error."""
"""Indicates that the server cannot or will not process the request due to something that is perceived to be a
client error"""


class UnauthorizedError(HTTPRequestStatusError):
"""A 401 error."""
"""Indicates that the client request has not been completed because it lacks valid authentication credentials for
the requested resource."""


class ForbiddenError(HTTPRequestStatusError):
"""A 403 error."""
"""Indicates that the server understands the request but refuses to authorize it, typically means that
permissions are missing for the resource requested."""


class NotFoundError(HTTPRequestStatusError):
"""A 404 error."""
"""Indicates that the server cannot find the requested resource."""


class InternalServerError(HTTPRequestStatusError):
"""A 5xx error."""
"""A 5xx error. Indicates that the server encountered an unexpected condition that prevented it from fulfilling
the request, this is typically not a user error."""


class CloudflareBanError(Exception):
"""A error for when you get banned by cloudflare
"""An error for when you get banned by cloudflare

This happens due to getting too many ``401``, ``403`` or ``429`` responses from discord.
This will block your access to the API temporarily for an hour.
Expand Down
7 changes: 4 additions & 3 deletions nextcore/http/file.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@

__all__: Final[tuple[str, ...]] = ("File",)

# This is not a attr.dataclass because it does not support slots.

# This is not an attr.dataclass because it does not support slots.
class File:
"""A utility class for uploading files to the API.

Expand All @@ -43,7 +44,7 @@ class File:
The name of the file.

.. warning::
Only files ending with a `supported file extension <https://discord.dev/reference#image-formatting-image-formats>`__ can be included in embeds.
Only files ending with a `supported file extension <https://discord.dev/reference#image-formatting-image-formats>`__ can be used in embeds.
contents:
The contents of the file.

Expand All @@ -53,7 +54,7 @@ class File:
The name of the file.

.. warning::
Only files ending with a `supported file extension <https://discord.dev/reference#image-formatting-image-formats>`__ can be included in embeds.
Only files ending with a `supported file extension <https://discord.dev/reference#image-formatting-image-formats>`__ can be used in embeds.
contents:
The contents of the file.
"""
Expand Down
25 changes: 13 additions & 12 deletions nextcore/http/rate_limit_storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ class RateLimitStorage:

Attributes
----------
global_lock:
The users per user global rate limit.
global_rate_limiter:
The users per-user global rate limit.
"""

__slots__ = ("_nextcore_buckets", "_discord_buckets", "_bucket_metadata", "global_rate_limiter")
Expand All @@ -69,29 +69,29 @@ def __init__(self) -> None:
# These are async and not just public dicts because we want to support custom implementations that use asyncio.
# This does introduce some overhead, but it's not too bad.
async def get_bucket_by_nextcore_id(self, nextcore_id: str) -> Bucket | None:
"""Get a rate limit bucket from a nextcore created id.
"""Gets a rate limit bucket from a nextcore created id.

Parameters
----------
nextcore_id:
The nextcore generated bucket id. This can be gotten by using :attr:`Route.bucket`
The nextcore generated bucket id. This can be retrieved by using :attr:`Route.bucket`
"""
return self._nextcore_buckets.get(nextcore_id)

async def store_bucket_by_nextcore_id(self, nextcore_id: str, bucket: Bucket) -> None:
"""Store a rate limit bucket by nextcore generated id.
"""Stores a rate limit bucket by nextcore generated id.

Parameters
----------
nextcore_id:
The nextcore generated id of the
The nextcore generated id of the bucket.
bucket:
The bucket to store.
"""
self._nextcore_buckets[nextcore_id] = bucket

async def get_bucket_by_discord_id(self, discord_id: str) -> Bucket | None:
"""Get a rate limit bucket from the Discord bucket hash.
"""Gets a rate limit bucket from the Discord bucket hash.

This can be obtained via the ``X-Ratelimit-Bucket`` header.

Expand All @@ -103,7 +103,7 @@ async def get_bucket_by_discord_id(self, discord_id: str) -> Bucket | None:
return self._discord_buckets.get(discord_id)

async def store_bucket_by_discord_id(self, discord_id: str, bucket: Bucket) -> None:
"""Store a rate limit bucket by the discord bucket hash.
"""Stores a rate limit bucket by the discord bucket hash.

This can be obtained via the ``X-Ratelimit-Bucket`` header.

Expand All @@ -117,7 +117,7 @@ async def store_bucket_by_discord_id(self, discord_id: str, bucket: Bucket) -> N
self._discord_buckets[discord_id] = bucket

async def get_bucket_metadata(self, bucket_route: str) -> BucketMetadata | None:
"""Get the metadata for a bucket from the route.
"""Gets the metadata for a bucket from the route.

Parameters
----------
Expand All @@ -127,7 +127,7 @@ async def get_bucket_metadata(self, bucket_route: str) -> BucketMetadata | None:
return self._bucket_metadata.get(bucket_route)

async def store_metadata(self, bucket_route: str, metadata: BucketMetadata) -> None:
"""Store the metadata for a bucket from the route.
"""Stores the metadata for a bucket from the route.

Parameters
----------
Expand All @@ -151,14 +151,15 @@ def _cleanup_buckets(self, phase: Literal["start", "stop"], info: dict[str, int]
for bucket_id, bucket in self._nextcore_buckets.copy().items():
if not bucket.dirty:
logger.debug("Cleaning up bucket %s", bucket_id)
# Delete the main reference. Other references like RateLimitStorage._discord_buckets should get cleaned up automatically as it is a weakref.
# Delete the main reference. Other references like RateLimitStorage._discord_buckets should get
# cleaned up automatically as it is a weakref.
del self._nextcore_buckets[bucket_id]

async def close(self) -> None:
"""Clean up before deletion.

.. warning::
If this is not called before you delete this or it goes out of scope, you will get a memory leak.
If this is not called before you delete this, or it goes out of scope; you will get a memory leak.
"""
# Remove the garbage collection callback
gc.callbacks.remove(self._cleanup_buckets)
Expand Down
6 changes: 5 additions & 1 deletion nextcore/http/request_session.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ class RequestSession:
If this request was made when the bucket was unlimited.

This exists to make sure that there is no bad state when switching between unlimited and limited.
priority:
The priority of the request (A lower number means it will be executed faster).

Attributes
----------
Expand All @@ -47,7 +49,9 @@ class RequestSession:

This exists to make sure that there is no bad state when switching between unlimited and limited.
pending_future:
The future that when set will execute the request.
The future that would be executed when set.
priority:
The priority of the request (A lower number means it will be executed faster).
"""

__slots__: Final[tuple[str, ...]] = ("pending_future", "priority", "unlimited")
Expand Down
Loading