Skip to content

Commit

Permalink
Minor refactoring around python comments
Browse files Browse the repository at this point in the history
  • Loading branch information
malhotrashivam committed Jan 29, 2024
1 parent 9847b59 commit 0750e56
Showing 1 changed file with 32 additions and 25 deletions.
57 changes: 32 additions & 25 deletions py/server/deephaven/experimental/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,31 +24,6 @@
class S3Instructions(JObjectWrapper):
"""
S3Instructions provides specialized instructions for reading from AWS S3.
Args:
aws_region_name (str): the AWS region name for reading parquet files stored in AWS S3, mandatory parameter.
max_concurrent_requests (int): the maximum number of concurrent requests for reading parquet files stored in S3.
default is 50.
read_ahead_count (int): the number of fragments to send asynchronous read requests for while reading the current
fragment. Defaults to 1, which means fetch the next fragment in advance when reading the current fragment.
fragment_size (int): the maximum size of each fragment to read from S3, defaults to 5 MB. If there are fewer
bytes remaining in the file, the fetched fragment can be smaller.
max_cache_size (int): the maximum number of fragments to cache in memory while reading, defaults to 32. This
caching is done at the deephaven layer for faster access to recently read fragments.
connection_timeout (Union[Duration, int, str, datetime.timedelta, np.timedelta64, pd.Timedelta]):
the amount of time to wait when initially establishing a connection before giving up and timing out, can be
expressed as an integer in nanoseconds, a time interval string, e.g. "PT00:00:00.001" or "PT1s", or other
time duration types. Default to 2 seconds.
read_timeout (Union[Duration, int, str, datetime.timedelta, np.timedelta64, pd.Timedelta]):
the amount of time to wait when reading a fragment before giving up and timing out, can be expressed as an
integer in nanoseconds, a time interval string, e.g. "PT00:00:00.001" or "PT1s", or other time duration
types. Default to 2 seconds.
aws_access_key_id (str): the AWS access key for reading parquet files stored in AWS S3. Both access key and
secret key must be provided to use static credentials, else default credentials will be used from
software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.
aws_secret_access_key (str): the AWS secret access key for reading parquet files stored in AWS S3. Both access
key and secret key must be provided to use static credentials, else default credentials will be used from
software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.
"""

j_object_type = _JS3Instructions
Expand All @@ -66,6 +41,38 @@ def __init__(self,
aws_access_key_id: Optional[str] = None,
aws_secret_access_key: Optional[str] = None):

"""
Initializes the instructions.
Args:
aws_region_name (str): the AWS region name for reading parquet files stored in AWS S3, mandatory parameter.
max_concurrent_requests (int): the maximum number of concurrent requests for reading parquet files stored in S3.
default is 50.
read_ahead_count (int): the number of fragments to send asynchronous read requests for while reading the current
fragment. Default to 1, which means fetch the next fragment in advance when reading the current fragment.
fragment_size (int): the maximum size of each fragment to read from S3, defaults to 5 MB. If there are fewer
bytes remaining in the file, the fetched fragment can be smaller.
max_cache_size (int): the maximum number of fragments to cache in memory while reading, defaults to 32. This
caching is done at the deephaven layer for faster access to recently read fragments.
connection_timeout (Union[Duration, int, str, datetime.timedelta, np.timedelta64, pd.Timedelta]):
the amount of time to wait when initially establishing a connection before giving up and timing out, can
be expressed as an integer in nanoseconds, a time interval string, e.g. "PT00:00:00.001" or "PT1s", or
other time duration types. Default to 2 seconds.
read_timeout (Union[Duration, int, str, datetime.timedelta, np.timedelta64, pd.Timedelta]):
the amount of time to wait when reading a fragment before giving up and timing out, can be expressed as
an integer in nanoseconds, a time interval string, e.g. "PT00:00:00.001" or "PT1s", or other time
duration types. Default to 2 seconds.
aws_access_key_id (str): the AWS access key for reading parquet files stored in AWS S3. Both access key and
secret key must be provided to use static credentials, else default credentials will be used from
software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.
aws_secret_access_key (str): the AWS secret access key for reading parquet files stored in AWS S3. Both
access key and secret key must be provided to use static credentials, else default credentials will be
used from software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.
Raises:
DHError: If unable to build the instructions object.
"""

try:
builder = self.j_object_type.builder()
builder.awsRegionName(aws_region_name)
Expand Down

0 comments on commit 0750e56

Please sign in to comment.