Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(db): add rollup indexes #558

Merged
merged 3 commits into from
Sep 10, 2024
Merged

perf(db): add rollup indexes #558

merged 3 commits into from
Sep 10, 2024

Conversation

PJColombo
Copy link
Member

@PJColombo PJColombo commented Sep 10, 2024

Checklist

  • My change requires a documentation update, and I have done it.
  • I have added tests to cover my changes.
  • I have filled out the description and linked the related issues.

Description

Resolves #547

This PR introduces the following indexes to the Transaction model:

  • (category, rollup, block_number, index): This index allows efficient filtering by category and rollup, while ensuring the results are sorted by block_number and index without needing an additional sorting step.
  • (fromId, block_number, index): Similar to the above, this index enables filtering by the transaction sender (fromId), while also returning results sorted by block_number and index.
  • (toId, block_number, index): Like the previous index, but this one allows filtering by the transaction receiver (toId), with results also sorted by block_number and index.

Motivation and Context (Optional)

Some filtering queries on the client have been experiencing timeouts, particularly when filtering blobs by rollup. This occurs because PostgreSQL needs to perform a full table scan to locate rows that match the selected rollup values.

Related Issue (Optional)

Screenshots (if appropriate):

Copy link

changeset-bot bot commented Sep 10, 2024

⚠️ No Changeset found

Latest commit: 0a261dd

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

vercel bot commented Sep 10, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
blobscan-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 10, 2024 0:16am
blobscan-staging ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 10, 2024 0:16am
4 Skipped Deployments
Name Status Preview Comments Updated (UTC)
blobscan-gnosis ⬜️ Ignored (Inspect) Sep 10, 2024 0:16am
blobscan-holesky ⬜️ Ignored (Inspect) Sep 10, 2024 0:16am
blobscan-mainnet ⬜️ Ignored (Inspect) Sep 10, 2024 0:16am
blobscan-sepolia ⬜️ Ignored (Inspect) Sep 10, 2024 0:16am

Copy link
Contributor

📦 Next.js Bundle Analysis for @blobscan/web

This analysis was generated by the Next.js Bundle Analysis action. 🤖

⚠️ Global Bundle Size Increased

Page Size (compressed)
global 283.64 KB (🟡 +38.02 KB)
Details

The global bundle is the javascript bundle that loads alongside every page. It is in its own category because its impact is much higher - an increase to its size means that every page on your website loads slower, and a decrease means every page loads faster.

Any third party scripts you have added directly to your app using the <script> tag are not accounted for in this analysis

If you want further insight into what is behind the changes, give @next/bundle-analyzer a try!

New Pages Added

The following pages were added to the bundle from the code in this PR:

Page Size (compressed) First Load % of Budget (350 KB)
/block_neighbor 251 B 283.88 KB 81.11%
/stats 347.49 KB 631.13 KB 180.32%

Eight Pages Changed Size

The following pages changed size from the code in this PR compared to its base branch:

Page Size (compressed) First Load % of Budget (350 KB)
/ 352.28 KB 635.92 KB 181.69% (🟢 -4.87%)
/address/[address] 27.29 KB 310.93 KB 88.84% (🟢 -4.70%)
/blob/[hash] 27.34 KB 310.98 KB 88.85% (🟢 -4.31%)
/blobs 75.95 KB 359.59 KB 102.74% (🟡 +12.32%)
/block/[id] 14.13 KB 297.77 KB 85.08% (🟢 -4.21%)
/blocks 73.67 KB 357.31 KB 102.09% (🟡 +12.31%)
/tx/[hash] 15.53 KB 299.17 KB 85.48% (🟡 +0.65%)
/txs 73.15 KB 356.78 KB 101.94% (🟡 +10.30%)
Details

Only the gzipped size is provided here based on an expert tip.

First Load is the size of the global bundle plus the bundle for the individual page. If a user were to show up to your website and land on a given page, the first load size represents the amount of javascript that user would need to download. If next/link is used, subsequent page loads would only need to download that page's bundle (the number in the "Size" column), since the global bundle has already been downloaded.

Any third party scripts you have added directly to your app using the <script> tag are not accounted for in this analysis

The "Budget %" column shows what percentage of your performance budget the First Load total takes up. For example, if your budget was 100kb, and a given page's first load size was 10kb, it would be 10% of your budget. You can also see how much this has increased or decreased compared to the base branch of your PR. If this percentage has increased by 20% or more, there will be a red status indicator applied, indicating that special attention should be given to this. If you see "+/- <0.01%" it means that there was a change in bundle size, but it is a trivial enough amount that it can be ignored.

Copy link

codecov bot commented Sep 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.37%. Comparing base (253f293) to head (0a261dd).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #558   +/-   ##
=======================================
  Coverage   89.37%   89.37%           
=======================================
  Files         150      150           
  Lines        9913     9913           
  Branches     1035     1035           
=======================================
  Hits         8860     8860           
  Misses       1053     1053           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@PJColombo PJColombo merged commit 359df4e into main Sep 10, 2024
14 checks passed
@PJColombo PJColombo deleted the feat/add-rollup-index branch September 10, 2024 00:20
@PabloCastellano
Copy link
Member

The issue is still present.

@PJColombo PJColombo restored the feat/add-rollup-index branch October 29, 2024 11:30
@PJColombo PJColombo deleted the feat/add-rollup-index branch October 29, 2024 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Timeout filtering blobs
2 participants