-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extract variable api from ScriptSession, let ScriptSession guard reads #4970
Merged
Merged
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
afcddf7
Extract variable api from ScriptSession, let ScriptSession guard reads
niloc132 4e2beee
Further shift from script session to query scope, including liveness
niloc132 7041746
Restore method needed for python local scope hack
niloc132 06ce7f0
Fix test setup, correct unwrapping in sql
niloc132 7b57fa2
Partial review feedback
niloc132 5a1a7cf
Remove VariableProvider, use QueryScope instead
niloc132 d309e35
Remove dead method, dead cleanup code in flight round trip test
niloc132 8b7c820
Remove more dead code, fix potential lock order issue
niloc132 bd2abcc
Field ordering
niloc132 8f8102e
Correct comment layout
niloc132 4efb27e
Review feedback:
niloc132 c049476
Revert attempt to use optional (incompatible with null returns)
niloc132 ff6a3ba
Merge branch 'main' into 4040-cleanup-5
niloc132 deb0a77
Review feedback
niloc132 c5c1ce7
Old requested rename, and added missing transferTo
niloc132 42ba9be
Finish implementation (don't trust IJ's refactor tools..?)
niloc132 0b6caa8
Review feedback
niloc132 d3f0472
Merge branch 'main' into 4040-cleanup-5
niloc132 e3a0583
Merge branch 'main' into 4040-cleanup-5
niloc132 cb8646d
Live code review
niloc132 abb9dcb
Live review feedback
niloc132 25e4da7
Clean up standalone impl
niloc132 7843871
Merge branch 'main' into 4040-cleanup-5
niloc132 d2208c3
spotless
niloc132 7598e14
Review feedback
niloc132 668596d
Review feedback
niloc132 1e32a0e
Add missing queryScope.unwrapObject() calls
niloc132 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Partial review feedback
- Loading branch information
commit 7b57fa217d8ad008b8807f0e0b86b85e04aff241
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As is, we end up making three copies of this key-set!
PyDictWrapper#keySet
creates a copy of the keys (instead of a view). Then we copy it a second time inPythonDeephavenSession#getVariableNames
using a collector. Finally, we copy it a third time inScriptSessionQueryScope#getParamNames
to filter out non-valid names.We should pass a
Predicate
togetVariableNames
so that the Groovy impl can filter under the param-map's object monitor. That makes python a two-copy impl and the groovy one a single-copy. (instead of three and two respectively)Do we need to synchronize w/
setVariable
to avoid changing the map whilePyDictWrapper#keySet
is making the initial copy? It looks like if you somesetVariable
to null while iterating that the result set could include the removed variable and not the variable that was next in the dict. Also, note thatPyListWrapper#iterator()
is not protected from size changes and will appears to return null for the last item in the dict (if the size is decreased).It looks like this would cause an NPE in the
LinkedHashSet
construction withinPyDictWrapper#keySet
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good points, will pass a predicate, stream results into the unmodifiable set.
Racing with setVariable is somewhat fraught - after all, setVariable isn't the only way to assign values into the collection, python itself could be assigning/removing something. I think technically I can use PyLib.ensureGil() here, to guarantee that reads of python structures can't change while reading, in place of the
synchronized
blocks that Groovy's impl uses). Any AbstractScriptSession override that makes more than one call into python should use this (and note that it is not safe to use this to call into arbitrary python, as that python code could end up dropping the GIL despite our efforts - ensureGil only lets us be sure that we have the GIL in Java, JNI, and our own C).