Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ga): sync ga data lambda #93

Merged
merged 16 commits into from
Nov 20, 2023
Merged

feat(ga): sync ga data lambda #93

merged 16 commits into from
Nov 20, 2023

Conversation

gary02
Copy link
Contributor

@gary02 gary02 commented Oct 19, 2023

No description provided.

Comment on lines +155 to +156
startDate,
endDate,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we need to check dates here? startDate <= endDate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invalid input would fail the GA4 api request

handlers/sync-ga4-data.ts Show resolved Hide resolved

export const getLocalDateString = (date: Date) => {
// return utc+8 date string in YYYY-MM-DD format
return date.toLocaleDateString("sv", { timeZone: "Asia/Taipei" });
Copy link
Contributor

@tx0c tx0c Nov 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this sv using a Swedish locale? maybe toISOString is better? does here require a UTC+8 timezone cutoff of each day?

> (new Date).toISOString().substring(0, 10)
'2023-11-14'

Copy link
Contributor Author

@gary02 gary02 Nov 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does here require a UTC+8 timezone cutoff of each day?

Yes, as GA4 timezone settings is UTC+8

}
}
if (insertRows.length > 0) {
await knex(TABLE_NAME).insert(insertRows);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and do you have estimation of how many rows will be there each day? what's growth estimation of this table per month;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and there's a way to upsert might be useful in the case, one line to handle all rows:

INSERT INTO table (...) VALUES (...)
ON CONFLICT (id)
DO UPDATE SET total_users=EXCLUDED.total_users ;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upsert way doesn't work here, because table article_ga4_data has an exclusion constraint which ON CONFLICT syntax doesn't support

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you have estimation of how many rows will be there each day? what's growth estimation of this table per month;

roughly 6k rows a day

@gary02 gary02 merged commit 595655e into main Nov 20, 2023
2 checks passed
@delete-merged-branch delete-merged-branch bot deleted the feat/sync-ga-data branch November 20, 2023 02:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants