virtualized list with react window & throttled the request to prevent 429 errors // add list counts to list view

[tb38r]

@noahm , addressed the virtualisation suggestion by implementing React Windows, only a subset of the lists are rendered at a time now. Additionally, used lodash's throttling functionality to minimise the frequency of the requests as it'd overload the servers.

Both appear fine, however, an issue remains in that the value returned useListSize is always '1'. That wasn't always the case as I'd initially tested the endpoint provided by @thieflord06 on Postman and would get varied results depending on the url provided however whether via Postman, or the app itself, it now always returns 1. Unsure why that may be, any suggestions?

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
clearsky-ui	❌ Failed (Inspect)			Jan 10, 2025 5:35pm

[thieflord06]

I will look into why the results have changed.

[thieflord06]

This is fixed now @tb38r

[thieflord06]

@tb38r We need to find a consistent spot for the list count number. Also, there is a view within the view.

Desktop:

Mobile:

[tb38r]

count now returns 150 on my end but your screenshots are closer to what's expected so I'll continue on the assumption that it might be a local issue.

will work on fixing the view within a view issue as well as applying some consistency amongst different the screen sizes.

[thieflord06]

Once we get the visuals sorted out we can verify the count logic. Staging is significantly behind prod so if you're comparing counts from in the bsky app, they will definitely be off.

I'm a little concerned about how many even numbered list counts there are (100, 50, 150, etc.) but I think people are also using tools to create lists so that kind of makes sense.

[noahm]

Off to a good start, but there's a few issues to fix. One is the double nested scroll areas which you're already aware of. Another is the fact that this has removed the behavior of loading additional pages of data once you scroll near to the bottom of the existing data.

[thieflord06]

This is shaping up good. There are some spacing issues with larger numbers.

[thieflord06]

Build failed

    at Module._compile (node:internal/modules/cjs/loader:1565:14)
    at Object..js (node:internal/modules/cjs/loader:1708:10)
    at Module.load (node:internal/modules/cjs/loader:1318:32)
    at Function._load (node:internal/modules/cjs/loader:1128:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:219:24)
    at cjsLoader (node:internal/modules/esm/translators:263:5)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:196:7) {
  [cause]: Error: Cannot find module '@rollup/rollup-linux-x64-gnu'
  Require stack:
  - /vercel/path0/node_modules/rollup/dist/native.js
      at Function._resolveFilename (node:internal/modules/cjs/loader:1249:15)
      at Function._load (node:internal/modules/cjs/loader:1075:27)
      at TracingChannel.traceSync (node:diagnostics_channel:322:14)
      at wrapModuleLoad (node:internal/modules/cjs/loader:219:24)
      at Module.require (node:internal/modules/cjs/loader:1340:12)
      at require (node:internal/modules/helpers:138:16)
      at requireWithFriendlyError (/vercel/path0/node_modules/rollup/dist/native.js:46:10)
      at Object.<anonymous> (/vercel/path0/node_modules/rollup/dist/native.js:73:76)
      at Module._compile (node:internal/modules/cjs/loader:1565:14)
      at Object..js (node:internal/modules/cjs/loader:1708:10) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [ '/vercel/path0/node_modules/rollup/dist/native.js' ]
  }
}
Node.js v22.12.0
Error: Command "npm run build" exited with 1

[thieflord06]

I think if we moved the list description to a new line and wrap the description that will allow for the list count to stay in the same spot and there won't be any spacing issues.

Being that there is a ticket out for this IDK if you want to do this in this PR or make a new one @tb38r

reference: #262

[tb38r]

Hey @thieflord06, apologies for going ghost, been tied up with various things as well as getting this issue done.

In trying to implement the windowing feature, the initial attempt via react-window added an additional scrollbar as previously highlighted where it'd be preferred to manage the browsing behaviour directly via the window. After a bit of research, found a comment from the package's creator noting it wouldn't be possible to do so with that specific package, was working perfectly otherwise - with the loading additional pages behaviour as well.

Found this --> https://tanstack.com/virtual/latest/docs/framework/react/examples/window which seemed perfect for the job. Spent some time trying to implement it but I'm possibly missing something minor to get it working perfectly

note the whitespace below, where the new data should be on scroll

const rowVirtualizer = useWindowVirtualizer({
count: filteredLists?.length,
estimateSize: () => 54,
overscan: 20,
gap: 50,
scrollMargin: listRef.current ? listRef.current.offsetTop : 0,
});

The result of --> rowVirtualizer.getVirtualItems().length represents the number of the items being rendered initially, working on figuring out why the rest of the data isn't showing, as I say, hoping finding solution involves a minor oversight on my end.

[tb38r]

branched off from list-counts, pushed updated work for review (branch 'window-throttling')

[thieflord06]

@tb38r No worries. I was just letting you know that your build is failing for some reason. maybe @noahm can give guidance on why.

I thought we may get this into the next release but it's ok if it doesn't make it because of the needed tweaks.

[noahm]

Your build issues are probably being caused by some strange changes I'm seeing in the package-lock.json file. If you revert that file to what is currently on dev branch, then run npm install, then commit the final state of the lockfile I expect things would be back into a good state.

[thieflord06]

@tb38r

#264 may affect your efforts.

[tb38r]

noted!

[thieflord06]

@tb38r You'll have to make a new PR for this if you want to keep working on it.