Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] unused SparkSession being import from pyspark.sql in several tests #139

Closed
dannymeijer opened this issue Nov 29, 2024 · 0 comments · Fixed by #140
Closed

[BUG] unused SparkSession being import from pyspark.sql in several tests #139

dannymeijer opened this issue Nov 29, 2024 · 0 comments · Fixed by #140
Labels
bug Something isn't working
Milestone
0.9.0

Comments

@dannymeijer
Copy link
Member 

          I see the SparkSession being import from pyspark.sql. This is also present in other test scripts (test_lookup, test_hana). Is this ok because locally we never rely on SparkConnect? Do you see a case in which we should set up the scripts so that they can be tested with both the sessions? (and dataframe classes of course)

Originally posted by @riccamini in #97 (comment)
@dannymeijer dannymeijer added this to the 0.9.0 milestone Nov 29, 2024
@dannymeijer dannymeijer added the bug Something isn't working label Nov 29, 2024
@dannymeijer dannymeijer moved this to In progress in Koheesio Nov 29, 2024
@dannymeijer dannymeijer mentioned this issue Nov 29, 2024
Merged
9 tasks
@dannymeijer dannymeijer linked a pull request Nov 29, 2024 that will close this issue
[FIX] unused SparkSession being import from pyspark.sql in several tests #140
Merged
9 tasks
@dannymeijer dannymeijer mentioned this issue Nov 29, 2024
Merged
dannymeijer added a commit that referenced this issue Nov 29, 2024
@dannymeijer
[FIX] unused SparkSession being import from pyspark.sql in several te…
5298c2b 
…sts (#140)

<!--- Provide a general summary of your changes in the Title above -->

## Description
<!--- Describe your changes in detail -->

## Related Issue
<!--- This project only accepts pull requests related to open issues -->
<!--- If suggesting a new feature or change, please discuss it in an
issue first -->
<!--- If fixing a bug, there should be an issue describing it with steps
to reproduce -->
<!--- Please link to the issue here: -->
#139

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->

## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Screenshots (if appropriate):

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)

## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes
that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [x] I have added tests to cover my changes.
- [x] All new and existing tests passed.

Co-authored-by: Danny Meijer <[email protected]>
@dannymeijer dannymeijer moved this from In progress to Done in Koheesio Nov 29, 2024
@dannymeijer dannymeijer closed this as completed by moving to Done in Koheesio Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Koheesio
Status: Done
Milestone
0.9.0
Development

Successfully merging a pull request may close this issue.

[FIX] unused SparkSession being import from pyspark.sql in several tests Nike-Inc/koheesio
1 participant
@dannymeijer