Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure pseudonymize_ne #58

Merged
merged 3 commits into from
Mar 3, 2025
Merged

Restructure pseudonymize_ne #58

merged 3 commits into from
Mar 3, 2025

Conversation

fexfl
Copy link
Collaborator

@fexfl fexfl commented Feb 16, 2025

Restructured pseudonymize_ne

  • Pseudonym of NE is now added to NE dict for all entity groups
  • Pseudonyms for LOC, ORG, MISC are just the usual brackets [...]
  • Pseudonym for PER is chosen in choose_per_pseudonym as previously done in pseudonymize_per
  • NE is replaced immediately, and only this exact occurence of the NE is replaced
  • All replaced NEs are recorded in self.ne_list with the chosen pseudonyms
  • Previously used self.per_list etc. were removed, as well as self.used_first_names

Tests and demo notebook were adjusted accordingly

@fexfl fexfl requested a review from iulusoy February 16, 2025 16:06
Copy link

codecov bot commented Feb 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.58%. Comparing base (d24d170) to head (bbcbe03).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #58      +/-   ##
==========================================
+ Coverage   93.90%   94.58%   +0.67%     
==========================================
  Files           4        4              
  Lines         394      406      +12     
==========================================
+ Hits          370      384      +14     
+ Misses         24       22       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fexfl
Copy link
Collaborator Author

fexfl commented Feb 16, 2025

This closes #54.

In first tests it also fixes the bug mentioned in #53.

@iulusoy iulusoy merged commit ae89a3c into main Mar 3, 2025
9 checks passed
@iulusoy iulusoy deleted the restructure-pseudo branch March 3, 2025 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants