Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/store user traffic #84

Merged
merged 8 commits into from
Nov 7, 2023
Merged

Conversation

FredrikMorstad
Copy link
Member

@FredrikMorstad FredrikMorstad commented Oct 22, 2023

Tracking of unique visits and page visits 🕵️

Unique visits

Anonymous tracking of unique visits per day
Uses bloom filter to check if a user has visited today. This allows users to be tracked without storing any user information meaning we don't have to have "terms and condition" in fronted.
After the day is over the bloom filter is removed from the database.

Mongodb timeseries

Uses time series to store the timestamp when the unique users was detected. Mongodb time series is specialised on storing these kinds of data and allow for fast queries and efficient storage.

Get unique visits

To allow minimal work for the frontend the endpoint gives the date with different intervals based on the range specified in the request. Meaning when the interval is under 48 hours the data is grouped to their hour when over the data is grouped to their day.

Page visits

Tracks visits on different pages on the website
The api blindly accepts a path and store it in a time series collection. This allows the fronted to send post request with different page visits.
As the fronted sends the location.pathname many pages will have path name contains their uuid. To allow the pages to be displayed with more readable names the api has function for finding the database object based on the path.

Most viewed pages last 30 days:

Endpoint for giving the 5 most viewed pages in the last 30 days

Resolves td-org-uit-no/tdctl-frontend#71

Something to consider 💡

Reddis cache

Setup reddis cache for the 30 most viewed page
- This cache can do the initial work of checking the most visited page at startup and then keep track in memory nd return fast results.

Stats service

  • Run the stats service on a separate server as an individual service with its own database avoiding load on the main API.
  • Since the stats api uses a lot common functionalities its a jobb to move the common functionalities to another repo and requires more maintenance.

@FredrikMorstad FredrikMorstad force-pushed the feature/store_user_traffic branch 2 times, most recently from 02bf98d to b401e7a Compare October 26, 2023 06:28
Copy link
Contributor

@otytlandsvik otytlandsvik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice stuff! 🚀 Left some nitpicky comments on a few things I think you should consider.

app/api/stats.py Outdated Show resolved Hide resolved
app/api/utils.py Outdated Show resolved Hide resolved
app/api/utils.py Outdated Show resolved Hide resolved
app/utils/date.py Show resolved Hide resolved
@FredrikMorstad FredrikMorstad force-pushed the feature/store_user_traffic branch from b401e7a to c5798b0 Compare November 5, 2023 20:21
Copy link
Contributor

@otytlandsvik otytlandsvik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@FredrikMorstad FredrikMorstad merged commit e5ea77b into master Nov 7, 2023
1 check passed
@FredrikMorstad FredrikMorstad deleted the feature/store_user_traffic branch November 7, 2023 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🕵️ Monitor website traffic
2 participants