Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to fetch Gsheet data using range #233

Merged
merged 1 commit into from
Aug 11, 2023
Merged

Add option to fetch Gsheet data using range #233

merged 1 commit into from
Aug 11, 2023

Conversation

Thootje
Copy link
Contributor

@Thootje Thootje commented Aug 11, 2023

Related to: #213

Finally came around to make this PR to add the option to fetch Google Sheet data using a range instead of fetching the entire sheet.

It works by calling .get(range) on the sheet instead of .get_all_records() if a range is provided. Within the source .yml file, you have to add a range using the range key and, optional, headers under the headers key you want to use in case your selected range doesn't have any headers.

version: 2

sources:
  - name: test_table
    schema: output
    meta:
      plugin: gsheet
      key: some_id_here
      worksheet: data
      range: A1:B2
      headers:
        - column_a
        - column_b
    tables:
      - name: test_table

Finally if you don't have a matching amount of headers and number of columns in the selected range, it will give a nice exception. For example, the below source .yml fetches 3 columns, but only 2 headers are provided. This will be caught by the exception.

version: 2

sources:
  - name: test_table
    schema: output
    meta:
      plugin: gsheet
      key: some_id_here
      worksheet: data
      range: A1:C2
      headers:
        - column_a
        - column_b
    tables:
      - name: test_table

@jwills jwills merged commit 1483bdf into duckdb:master Aug 11, 2023
31 of 32 checks passed
@jwills
Copy link
Collaborator

jwills commented Aug 11, 2023

Thank you @Thootje!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants