Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EW-1019: TSP Testing #5375

Open
wants to merge 43 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
0ce5351
Add empty line.
mkreuzkam-cap Dec 4, 2024
7bbabe7
log token
mkreuzkam-cap Dec 4, 2024
fcd8d6d
Remove pLimit temporarily for testing.
mkreuzkam-cap Dec 9, 2024
49bfdda
Manual batching of data.
mkreuzkam-cap Dec 9, 2024
628d8c3
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 9, 2024
397dff6
Fix array size.
mkreuzkam-cap Dec 9, 2024
e4548fc
Better logging.
mkreuzkam-cap Dec 9, 2024
634049c
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 9, 2024
0f7acca
Even more logs!
mkreuzkam-cap Dec 9, 2024
1c342a0
Logging of errors.
mkreuzkam-cap Dec 10, 2024
a9669a5
Use bulk operations for sync. (Very WIP!!!)
mkreuzkam-cap Dec 11, 2024
3633cc0
EW-1019 Fix Linter warnings
SimoneRadtke-Cap Dec 11, 2024
ac863a7
EW-1019 Fix linter warnings
SimoneRadtke-Cap Dec 11, 2024
551f0e6
Use bulk operations for migration.
mkreuzkam-cap Dec 11, 2024
3c805b4
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 11, 2024
79c6a7c
EW-1019 Add docker compose file to gitignore
SimoneRadtke-Cap Dec 11, 2024
18217e6
fix linter warning
mkreuzkam-cap Dec 11, 2024
643c843
Bulk migration for students.
mkreuzkam-cap Dec 13, 2024
0a38d25
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 16, 2024
a04d22d
Use configService directly and don't read values in constructor.
mkreuzkam-cap Dec 16, 2024
c16a7bf
Add batching.
mkreuzkam-cap Dec 16, 2024
92e1b49
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 17, 2024
7dc774f
remove console log, add comments
mkreuzkam-cap Dec 17, 2024
2587c90
Clean up migration and move into service class.
mkreuzkam-cap Dec 17, 2024
470329c
Clean up loggables.
mkreuzkam-cap Dec 17, 2024
4a163ae
EW.1019 Add saveAll to account-idm-service
SimoneRadtke-Cap Dec 17, 2024
29534a2
adjust config var usage.
mkreuzkam-cap Dec 17, 2024
8d19ebf
adjust tests (wip)
mkreuzkam-cap Dec 17, 2024
a34c3a8
Fix findMany test in accoundIdm service.
mkreuzkam-cap Dec 18, 2024
1b0627b
Fix tsp sync strategy test.
mkreuzkam-cap Dec 18, 2024
216dfb0
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 18, 2024
c734169
Add tests for user service and repo.
mkreuzkam-cap Dec 18, 2024
91d5632
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 18, 2024
54a3d91
EW-1019 Adjust tsp sync service test
SimoneRadtke-Cap Dec 18, 2024
a6e97fa
Fix tests which broke for some reason.
mkreuzkam-cap Dec 18, 2024
b9edb39
Fix another test which broke for no reason.
mkreuzkam-cap Dec 18, 2024
9f3e9f1
Add tests for accounts.
mkreuzkam-cap Dec 19, 2024
e64b935
Fix an oopsie.
mkreuzkam-cap Dec 19, 2024
c58364b
EW-1019 Add test
SimoneRadtke-Cap Dec 19, 2024
905685d
Finish tsp sync migration service test.
mkreuzkam-cap Dec 19, 2024
b49aedb
Merge branch 'main' into EW-1019
mkreuzkam-cap Dec 19, 2024
d534101
Reduce amount of awaits.
mkreuzkam-cap Dec 19, 2024
27c73ea
remove async/await.
mkreuzkam-cap Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,6 @@ build
/.idea/
/apps/server/src/modules/board/loadtest/**/*.html
/apps/server/src/modules/board/loadtest/artilleryreport.json

# Docker
docker-compose.yml
10 changes: 9 additions & 1 deletion apps/server/src/infra/sync/sync.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import { TspSyncService } from './tsp/tsp-sync.service';
import { TspSyncStrategy } from './tsp/tsp-sync.strategy';
import { SyncUc } from './uc/sync.uc';
import { TspFetchService } from './tsp/tsp-fetch.service';
import { TspSyncMigrationService } from './tsp/tsp-sync-migration.service';

@Module({
imports: [
Expand All @@ -41,7 +42,14 @@ import { TspFetchService } from './tsp/tsp-fetch.service';
SyncUc,
SyncService,
...((Configuration.get('FEATURE_TSP_SYNC_ENABLED') as boolean)
? [TspSyncStrategy, TspSyncService, TspOauthDataMapper, TspFetchService, TspLegacyMigrationService]
? [
TspSyncStrategy,
TspSyncService,
TspOauthDataMapper,
TspFetchService,
TspLegacyMigrationService,
TspSyncMigrationService,
]
: []),
],
exports: [SyncConsole],
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { Loggable, LogMessage } from '@src/core/logger';

export class TspLegacyMigrationStartLoggable implements Loggable {
getLogMessage(): LogMessage {
public getLogMessage(): LogMessage {
const message: LogMessage = {
message: 'Running migration of legacy tsp data.',
};
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { Loggable, LogMessage } from '@src/core/logger';

export class TspLegacyMigrationSystemMissingLoggable implements Loggable {
getLogMessage(): LogMessage {
public getLogMessage(): LogMessage {
const message: LogMessage = {
message: 'No legacy system found',
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { Loggable, LogMessage } from '@src/core/logger';
export class TspLegacySchoolMigrationCountLoggable implements Loggable {
constructor(private readonly total: number) {}

getLogMessage(): LogMessage {
public getLogMessage(): LogMessage {
const message: LogMessage = {
message: `Found ${this.total} legacy tsp schools to migrate`,
data: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { Loggable, LogMessage } from '@src/core/logger';
export class TspLegacySchoolMigrationSuccessLoggable implements Loggable {
constructor(private readonly total: number, private readonly migrated: number) {}

getLogMessage(): LogMessage {
public getLogMessage(): LogMessage {
const message: LogMessage = {
message: `Legacy tsp data migration finished. Total schools: ${this.total}, migrated schools: ${this.migrated}`,
data: {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { TspMigrationBatchSummaryLoggable } from './tsp-migration-batch-summary.loggable';

describe(TspMigrationBatchSummaryLoggable.name, () => {
let loggable: TspMigrationBatchSummaryLoggable;

beforeAll(() => {
loggable = new TspMigrationBatchSummaryLoggable(1, 2, 3, 4, 5);
});

describe('when loggable is initialized', () => {
it('should be defined', () => {
expect(loggable).toBeDefined();
});
});

describe('getLogMessage', () => {
it('should return a log message', () => {
expect(loggable.getLogMessage()).toEqual({
message: `Migrated 2 users and 3 accounts in batch of size 1 (total done: 4, total migrations: 5)`,
data: {
batchSize: 1,
usersUpdated: 2,
accountsUpdated: 3,
totalDone: 4,
totalMigrations: 5,
},
});
});
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import { Loggable, LogMessage } from '@src/core/logger';

export class TspMigrationBatchSummaryLoggable implements Loggable {
constructor(
private readonly batchSize: number,
private readonly usersUpdated: number,
private readonly accountsUpdated: number,
private readonly totalDone: number,
private readonly totalMigrations: number
) {}

public getLogMessage(): LogMessage {
const message: LogMessage = {
message: `Migrated ${this.usersUpdated} users and ${this.accountsUpdated} accounts in batch of size ${this.batchSize} (total done: ${this.totalDone}, total migrations: ${this.totalMigrations})`,
data: {
batchSize: this.batchSize,
usersUpdated: this.usersUpdated,
accountsUpdated: this.accountsUpdated,
totalDone: this.totalDone,
totalMigrations: this.totalMigrations,
},
};

return message;
}
}
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import { TspStudentsFetchedLoggable } from './tsp-students-fetched.loggable';
import { TspMigrationsFetchedLoggable } from './tsp-migrations-fetched.loggable';

describe(TspStudentsFetchedLoggable.name, () => {
let loggable: TspStudentsFetchedLoggable;
describe(TspMigrationsFetchedLoggable.name, () => {
let loggable: TspMigrationsFetchedLoggable;

beforeAll(() => {
loggable = new TspStudentsFetchedLoggable(10);
loggable = new TspMigrationsFetchedLoggable(10);
});

describe('when loggable is initialized', () => {
Expand All @@ -16,9 +16,9 @@ describe(TspStudentsFetchedLoggable.name, () => {
describe('getLogMessage', () => {
it('should return a log message', () => {
expect(loggable.getLogMessage()).toEqual({
message: `Fetched 10 students for migration from TSP`,
message: `Fetched 10 users for migration from TSP`,
data: {
tspStudentCount: 10,
tspUserMigrationCount: 10,
},
});
});
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { Loggable, LogMessage } from '@src/core/logger';

export class TspMigrationsFetchedLoggable implements Loggable {
constructor(private readonly tspUserMigrationCount: number) {}

public getLogMessage(): LogMessage {
const message: LogMessage = {
message: `Fetched ${this.tspUserMigrationCount} users for migration from TSP`,
data: {
tspUserMigrationCount: this.tspUserMigrationCount,
},
};

return message;
}
}

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ describe(TspUsersMigratedLoggable.name, () => {
let loggable: TspUsersMigratedLoggable;

beforeAll(() => {
loggable = new TspUsersMigratedLoggable(10);
loggable = new TspUsersMigratedLoggable(10, 5, 4);
});

describe('when loggable is initialized', () => {
Expand All @@ -16,9 +16,11 @@ describe(TspUsersMigratedLoggable.name, () => {
describe('getLogMessage', () => {
it('should return a log message', () => {
expect(loggable.getLogMessage()).toEqual({
message: `Migrated users: 10 users migrated`,
message: `Migrated 5 users and 4 accounts. Total amount of migrations requested: 10`,
data: {
migratedUsers: 10,
totalMigrations: 10,
migratedUsers: 5,
migratedAccounts: 4,
},
});
});
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
import { Loggable, LogMessage } from '@src/core/logger';

export class TspUsersMigratedLoggable implements Loggable {
constructor(private readonly migratedUsers: number) {}
constructor(
private readonly totalMigrations: number,
private readonly migratedUsers: number,
private readonly migratedAccounts: number
) {}

public getLogMessage(): LogMessage {
const message: LogMessage = {
message: `Migrated users: ${this.migratedUsers} users migrated`,
message: `Migrated ${this.migratedUsers} users and ${this.migratedAccounts} accounts. Total amount of migrations requested: ${this.totalMigrations}`,
data: {
totalMigrations: this.totalMigrations,
migratedUsers: this.migratedUsers,
migratedAccounts: this.migratedAccounts,
},
};

Expand Down
Loading
Loading