Refactoring and fixing issues with job requests types #277
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixing logic where pip installs the updated packages (to avoid the entire image rebuild), so make sure deps are ignored (as this was the slow part).
Adding type GET_SERIALIZED_FORM to get the entire serialized state of a dataset.
Refactor to ensure it takes advantage of connection handling via its async context manager logic.
Add new function to get serialized dataset details to DatasetExternalClient.
Fix issue with not acquiring a session, and wrapping some things in try block to catch and log exceptions.
Fixing bug in response to LIST_FILES query, where reason text was not being assembled entirely correctly.
Adding optional offset and length params to abstract interface definition of get_data, and adding get_file_stat abstract method.
Adding support for querying about dataset items.
Updating to support indicating start and size of data for partial transfers.
Optimizing the reloading of datasets on object store manager startup.
Accounting for changes to get_data parameters and implementing get_file_stat.
Updating service to respond to GET_DATASET_ITEMS queries and requests for data with an offset start (i.e. partials).
Adding initial second view display and manage existing datasets, along with navigation functionality to toggle between "manage" and "create" views.
Sending serialized datasets to HTML template as list/array rather than dict/map.
Implementing layout and initial details-viewing behavior for dataset management GUI view.
Fixing argument handling for those args related to communicating with evaluation service, including the dest name for args, defaults, and use when creating main service handler class.
Fixing imports, class usage, and code style spacing for several things related to evaluation requests within main request-service handler class.
Adding DataFormat.can_format_fulfill class method to test if alternative data format is compatible, and updating logic of DataDomain.contains method to test and account for this type of compatibility.
Updating dmod.dataservice to depend on updated dmod.core (>=0.5.0).
Updating service manager find_dataset_for_requirement method to account for potentially compatible alternative dataset formats and examine any datasets having such formats if there are no matches with exactly the specified format.
Fixing bugs in AbstractInitRequest and DmodJobRequest types, where both the function and the nested call to the superclass function were __int__ instead of __init__.
Modifying class supertype, then updating __init__ and adding necessary properties.
- Updating class __init__ to require params previously taken from (already present) job_request param to conform to standard pattern used in class hierarchies - Adding convenience class method to factory create an instance with just the originating message, which __init__ previously did - Updating deserialization to account for above changes
Updating interface definition and existing implementation of job manager create_job function to have a fixed, required 'request' parameter of a type extending DmodJobRequest.
Updating dmod.client to depend on dmod.communication>=0.11.0.
Updating dmod.requestservice to depend on dmod.communication>=0.11.0.
Update to new dmod.communication (0.11.0) and dmod.scheduler (0.10.0).
Updating to depend on new dmod.communication 0.11.0.
Updating to depend on dmod.scheduler 0.10.0.
This was referenced Mar 14, 2023
|
Closing; replaced by #300, which isolates the relevant changes. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
create_jobin JobManager interface and implementations to expect a DmodJobRequestThe branch for this PR is based on the branch in #276; as such, that PR blocks this one.