Skip to content

Commit

Permalink
CTOOLS-348: allow timeout and rate limit retry configuration
Browse files Browse the repository at this point in the history
  • Loading branch information
@cg-finbourne
cg-finbourne committed Sep 23, 2024
1 parent e82328e commit 7c5dda4
Show file tree
Hide file tree
Showing 20 changed files with 1,579 additions and 151 deletions.
3 changes: 3 additions & 0 deletions generate/config-template.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@
},
"extensions/file_access_token.mustache": {
"destinationFilename": "${PACKAGE_NAME}/extensions/file_access_token.py"
},
"extensions/configuration_options.mustache": {
"destinationFilename": "${PACKAGE_NAME}/extensions/configuration_options.py"
}
}
}
33 changes: 32 additions & 1 deletion generate/templates/README.mustache
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ import {{{packageName}}}

## Getting Started

You'll need to provide some configuration to connect to the {{appName}} - see the articles about [short-lived access tokens](https://support.lusid.com/knowledgebase/article/KA-01654) and a [long-lived Personal Access Token](https://support.lusid.com/knowledgebase/article/KA-01774). This configuration can be provided using a secrets file or environment variables.
You'll need to provide some configuration to connect to the {{appName}} - see the articles about [short-lived access tokens](https://support.lusid.com/knowledgebase/article/KA-01654) and a [long-lived Personal Access Token](https://support.lusid.com/knowledgebase/article/KA-01774). This configuration can be provided using a secrets file or environment variables.

For some configuration it is also possible to override the global configuration at the ApiClientFactory level, or at the request level. For the set of configuration which can be overridden, please see [ConfigurationOptions](sdk/{{packageName}}/extensions/configuration_options.py). For a code illustration of this configuration being overridden, please see the [example](#example).

### Environment variables

Expand All @@ -78,6 +80,18 @@ FBN_ACCESS_TOKEN

You can send your requests to the {{appName}} via a proxy, by setting `FBN_PROXY_ADDRESS`. If your proxy has basic auth enabled, you must also set `FBN_PROXY_USERNAME` and `FBN_PROXY_PASSWORD`.

Other optional configuration

```bash
# sdk client timeouts in milliseconds - a value of 0 means no timeout, otherwise timeout values must be a positive integer
# please note - the chances of seeing a network issue increase with the duration of the request
# for this reason, rather than increasing the timeout, it will be more reliable to use an alternate polling style endpoint where these exist
FBN_TOTAL_TIMEOUT_MS # the default is 1800000 (30 minutes)
FBN_CONNECT_TIMEOUT_MS # the default is 0 (no timeout)
FBN_READ_TIMEOUT_MS # the default is 0 (no timeout)
FBN_RATE_LIMIT_RETRIES # the default is 2
```

### Secrets file

The secrets file must be in the current working directory.
Expand Down Expand Up @@ -126,6 +140,23 @@ You can send your requests to the {{appName}} via a proxy, by adding a proxy sec
}
```

Other optional configuration

```javascript
{
"api":
{
// sdk client timeouts in milliseconds - a value of 0 means no timeout, otherwise timeout values must be a positive integer
// please note - the chances of seeing a network issue increase with the duration of the request
// for this reason, rather than increasing the timeout, it will be more reliable to use an alternate polling style endpoint where these exist
"totalTimeoutMs":<timeout-in-ms>, // the default is 1800000 (30 minutes)
"connectTimeoutMs":<timeout-in-ms>, // the default is 0 (no timeout)
"readTimeoutMs":<timeout-in-ms>, // the default is 0 (no timeout)
"rateLimitRetries":<retries-when-being-rate-limited> // the default is 2
}
}
```

{{#apiInfo}}{{#apis}}{{#-first}}{{#operations}}{{#operation}}{{#-first}}### Example
{{> example}}
{{/-first}}{{/operation}}{{/operations}}{{/-first}}{{/apis}}{{/apiInfo}}
Expand Down
19 changes: 10 additions & 9 deletions generate/templates/api.mustache
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ from {{packageName}}.exceptions import ( # noqa: F401
ApiTypeError,
ApiValueError
)
from {{packageName}}.extensions.configuration_options import ConfigurationOptions


{{#operations}}
Expand Down Expand Up @@ -66,10 +67,9 @@ class {{classname}}:
{{/allParams}}
:param async_req: Whether to execute the request asynchronously.
:type async_req: bool, optional
:param _request_timeout: timeout setting for this request.
If one number provided, it will be total request
timeout. It can also be a pair (tuple) of
(connection, read) timeouts.
:param _request_timeout: Timeout setting. Do not use - use the opts parameter instead
:param opts: Configuration options for this request
:type opts: ConfigurationOptions, optional
:return: Returns the result object.
If the method is called asynchronously,
returns the request thread.
Expand Down Expand Up @@ -112,10 +112,9 @@ class {{classname}}:
:param _return_http_data_only: response data instead of ApiResponse
object with status code, headers, etc
:type _return_http_data_only: bool, optional
:param _request_timeout: timeout setting for this request. If one
number provided, it will be total request
timeout. It can also be a pair (tuple) of
(connection, read) timeouts.
:param _request_timeout: Timeout setting. Do not use - use the opts parameter instead
:param opts: Configuration options for this request
:type opts: ConfigurationOptions, optional
:param _request_auth: set to override the auth_settings for an a single
request; this effectively ignores the authentication
in the spec for a single request.
Expand Down Expand Up @@ -162,7 +161,8 @@ class {{classname}}:
'_request_timeout',
'_request_auth',
'_content_type',
'_headers'
'_headers',
'opts'
]
)

Expand Down Expand Up @@ -304,6 +304,7 @@ class {{classname}}:
_return_http_data_only=_params.get('_return_http_data_only'), # noqa: E501
_preload_content=_params.get('_preload_content', True),
_request_timeout=_params.get('_request_timeout'),
opts=_params.get('opts'),
{{#servers.0}}
_host=_host,
{{/servers.0}}
Expand Down
43 changes: 26 additions & 17 deletions generate/templates/api_client.mustache
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ class ApiClient:
files=None, response_types_map=None, auth_settings=None,
_return_http_data_only=None, collection_formats=None,
_preload_content=True, _request_timeout=None, _host=None,
_request_auth=None):
_request_auth=None, opts=None):

config = self.configuration

Expand Down Expand Up @@ -230,7 +230,8 @@ class ApiClient:
headers=header_params,
post_params=post_params, body=body,
_preload_content=_preload_content,
_request_timeout=_request_timeout)
_request_timeout=_request_timeout,
opts=opts)
except ApiException as e:
if e.body:
e.body = e.body.decode('utf-8')
Expand Down Expand Up @@ -390,7 +391,8 @@ class ApiClient:
response_types_map=None, auth_settings=None,
async_req=None, _return_http_data_only=None,
collection_formats=None, _preload_content=True,
_request_timeout=None, _host=None, _request_auth=None):
_request_timeout=None, _host=None, _request_auth=None,
opts=None):
"""Makes the HTTP request (synchronous) and returns deserialized data.

To make an async_req request, set the async_req parameter.
Expand All @@ -417,10 +419,9 @@ class ApiClient:
Default is True.
:param collection_formats: dict of collection formats for path, query,
header, and post parameters.
:param _request_timeout: timeout setting for this request. If one
number provided, it will be total request
timeout. It can also be a pair (tuple) of
(connection, read) timeouts.
:param _request_timeout: Timeout setting. Do not use - use the opts parameter instead
:param opts: Configuration options for this request
:type opts: ConfigurationOptions, optional
:param _request_auth: set to override the auth_settings for an a single
request; this effectively ignores the authentication
in the spec for a single request.
Expand All @@ -439,7 +440,7 @@ class ApiClient:
response_types_map, auth_settings,
_return_http_data_only, collection_formats,
_preload_content, _request_timeout, _host,
_request_auth)
_request_auth, opts)

return self.pool.apply_async(self.__call_api, (resource_path,
method, path_params,
Expand All @@ -452,61 +453,69 @@ class ApiClient:
collection_formats,
_preload_content,
_request_timeout,
_host, _request_auth))
_host, _request_auth,
opts))

def request(self, method, url, query_params=None, headers=None,
post_params=None, body=None, _preload_content=True,
_request_timeout=None):
_request_timeout=None, opts=None):
"""Makes the HTTP request using RESTClient."""
if method == "GET":
return self.rest_client.get_request(url,
query_params=query_params,
_preload_content=_preload_content,
_request_timeout=_request_timeout,
headers=headers)
headers=headers,
opts=opts)
elif method == "HEAD":
return self.rest_client.head_request(url,
query_params=query_params,
_preload_content=_preload_content,
_request_timeout=_request_timeout,
headers=headers)
headers=headers,
opts=opts)
elif method == "OPTIONS":
return self.rest_client.options_request(url,
query_params=query_params,
headers=headers,
_preload_content=_preload_content,
_request_timeout=_request_timeout)
_request_timeout=_request_timeout,
opts=opts)
elif method == "POST":
return self.rest_client.post_request(url,
query_params=query_params,
headers=headers,
post_params=post_params,
_preload_content=_preload_content,
_request_timeout=_request_timeout,
body=body)
body=body,
opts=opts)
elif method == "PUT":
return self.rest_client.put_request(url,
query_params=query_params,
headers=headers,
post_params=post_params,
_preload_content=_preload_content,
_request_timeout=_request_timeout,
body=body)
body=body,
opts=opts)
elif method == "PATCH":
return self.rest_client.patch_request(url,
query_params=query_params,
headers=headers,
post_params=post_params,
_preload_content=_preload_content,
_request_timeout=_request_timeout,
body=body)
body=body,
opts=opts)
elif method == "DELETE":
return self.rest_client.delete_request(url,
query_params=query_params,
headers=headers,
_preload_content=_preload_content,
_request_timeout=_request_timeout,
body=body)
body=body,
opts=opts)
else:
raise ApiValueError(
"http method must be `GET`, `HEAD`, `OPTIONS`,"
Expand Down
Loading

0 comments on commit 7c5dda4

Please sign in to comment.