Wrote a blog post on updating sort and dist keys in Redshift #7
Conversation
|
|
||
| ## Understanding the Problem | ||
|
|
||
| If your reading this post then you ran into the same blockage as many data engineers have when using AWS Redshift. I assume you followed the following steps: created an instance of Redshift to house all your data for your analytics, leveraged a third party service to ELT most of your data and allowed engineers to also dump data into redshift. Now, all data is following into Redshift and all analytics data is being accessed through Redshift. This is working great but you begin to see degradation in speed. You look into your initial setup and you realize that you are not leveraging key features of Redshift, the sortkey and distkey (Check out [this article](https://www.periscopedata.com/blog/double-your-redshift-performance-with-the-right-sortkeys-and-distkeys.html) to see the improvements with sortkeys/distkeys). |
There was a problem hiding this comment.
If you're reading this post then you ran into the same blockage as many data engineers have when using AWS Redshift. I assume you followed the following steps: created an instance of Redshift to house all your data for your analytics, leveraged a third party service to ETL most of your data, and allowed engineers to also dump data into redshift. Now all data, including analytics, is being accessed through Redshift. This works great but you begin to see degradation in speed. You look into your initial setup and realize that you are not leveraging key features of Redshift, the (bold sortkey and distkey to help with emphasis -->) sortkey and distkey (Check out this article to see the improvements with sortkeys/distkeys).
|
Please replace actual column names and structure with something more blog-friendly: widgets and gizmos (or posts and comments) vs our actual structure. |
|
I will update the column names and update the picture and create a new pull request. |
This is a blog post that shows people how to work around redshift restrictions with updating or adding thier sort and dist keys.