Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce field length to fit in BQ #63

Merged
merged 1 commit into from
Jul 16, 2024

Conversation

sydneynotthecity
Copy link
Contributor

@sydneynotthecity sydneynotthecity commented Jul 16, 2024

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with the jira ticket associated with the PR.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated the docs and README with the added features, breaking changes, new instructions on how to use the repository.

Release planning

  • I've decided if this PR requires a new major/minor/patch version accordingly to
    semver, and I've changed the name of the BRANCH to release/* , feature/* or patch/* .

What

The current dbt build is broken and cannot release or build a new docker image because field descriptions violate BigQuery field description limitations:

�[0m18:30:39  �[33m400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/test-hubble-319619/datasets/dbt_jenkins_crypto_stellar_dbt/tables/config_settings_current?prettyPrint=false: The description for field config_setting_id is too long. The maximum length is 1024 characters.�[0m
�[0m18:30:39  
�[0m18:30:39  �[33m400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/test-hubble-319619/datasets/dbt_jenkins_crypto_stellar_dbt/tables/accounts_current?prettyPrint=false: The description for field flags is too long. The maximum length is 1024 characters.�[0m
�[0m18:30:39  
�[0m18:30:39  �[33m400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/test-hubble-319619/datasets/dbt_jenkins_crypto_stellar_dbt/tables/trust_lines_current?prettyPrint=false: The description for field flags is too long. The maximum length is 1024 characters.�[0m

This PR resolves the violations by shortening the markdown used to generate tables. The change is visually ugly, but gets the fields under the 1024 character limit.

Why

The stellar-dbt repo persists docs to BigQuery and pushes any updates through this configuration: https://github.com/stellar/stellar-dbt/blob/master/dbt_project.yml#L68 The pipeline is broken because dbt is unable to persist the docs.

Known limitations

[TODO or N/A]

@sydneynotthecity sydneynotthecity requested a review from a team as a code owner July 16, 2024 19:51
@sydneynotthecity sydneynotthecity merged commit ff56fbc into master Jul 16, 2024
3 checks passed
@sydneynotthecity sydneynotthecity deleted the patch/shorten-column-descriptions branch November 14, 2024 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants