lifinity remove pools CTE

[andrewhong5297]

pools inner join on token accounts fungible was wrong, pools were actually NFTs. I went ahead and just removed the whole pools CTE since we don't need it anymore as the data is in transfers. this should run much faster now.

[andrewhong5297]

once this runs, we should check the CI output against this query to see if there are missing trades. honestly should maybe generally have a test that checks for tx_id from decoded table in final trades table
https://dune.com/queries/3905260

[jeff-dude]

is the issue here just being blocked by CI running? if so, we can look to increase the size and try again.

once this runs, we should check the CI output against this query to see if there are missing trades. honestly should maybe generally have a test that checks for tx_id from decoded table in final trades table https://dune.com/queries/3905260

this may be a bit difficult, as we rename CI tables to be unique and you'd want to lookup to the CI table you just built for the test. for now, it may be best to just do this type of testing during the development phase.

[andrewhong5297]

Yeah CI is timing out but this should otherwise be fine.

[jeff-dude]

Yeah CI is timing out but this should otherwise be fine.

hmm slightly concerning that it still times out on a cluster more than double the size. i will have to try again early next week and monitor closely. also will be curious to see history of these two full refreshes on old code, which in theory should be slower since it had more joins and such.

[andrewhong5297]

Wait we never merged this!! This is a big bug we need this merged

[jeff-dude]

Wait we never merged this!! This is a big bug we need this merged

this one was still timing out on CI cluster after i made it the size of prod, which isn't a great sign. i can give it another go today and hopefully see it run.

[jeff-dude]

it worked! phew. i feel more confident pushing this through now.

[andrewhong5297]

Haha awesome thats a relief! Thanks for trying it