Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement kmeans with kmeans++ #24

Closed
wants to merge 12 commits into from
Closed

Conversation

vberezhnev
Copy link
Contributor

No description provided.

@vberezhnev vberezhnev force-pushed the feature/mutual-scores-kmeans branch 2 times, most recently from a935134 to 5dba760 Compare October 30, 2024 03:33
Copy link
Contributor

@ichorid ichorid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add tests and remove CI/CD stuff (that is subject to a separate PR). Also, move the k-means algorithm into a separate file.

@vberezhnev
Copy link
Contributor Author

@automainint What do you think about moving files like clustering.rs, log.rs, and astar.rs into a separate folder specifically for libraries?

@automainint
Copy link
Collaborator

@automainint What do you think about moving files like clustering.rs, log.rs, and astar.rs into a separate folder specifically for libraries?

I usually avoid adding abstractions without a specific reason. E.g. protocol.rs is a separate file because it's used by the psql-connector, while some other parts of the service are not. protocol.rs depends on log.rs, but not on astar.rs, so log.rs and astar.rs have distinct status, and it's not obvious that they have to be grouped together.

Copy link
Collaborator

@automainint automainint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each function that returns a score should also return the cluster value for that score. So we have to change function signatures, and it's probably better to apply (or read cached) clusters in functions fetch_score and fetch_all_scores.

Currently read_mutual_scores doesn't apply clustering correctly. All scores from a user node should be clustered, not all scores to a user node.

I will change function signatures and write tests in the new branch feature/score_clustering.

Also we will have to implement caching of clusters as "group bounds", as mentioned in #10 (comment) by @ichorid.

@automainint automainint mentioned this pull request Nov 13, 2024
8 tasks
@automainint automainint deleted the feature/mutual-scores-kmeans branch November 17, 2024 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add normalization of mutual stats to meritrank-service "MUTUAL_SCORES"
3 participants