Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EW-1047: Add TSP System on Dev Environments #5309

Merged
merged 34 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
677c792
EW-1047: Add TSP System on Dev Environments
mkreuzkam-cap Oct 23, 2024
cafc09c
Merge branch 'main' into EW-1047
mkreuzkam-cap Oct 23, 2024
98063b3
include dev dependencies to run nest cli.
mkreuzkam-cap Oct 23, 2024
f0285a3
Use encryption service to decrypt client secret if AES key is set.
mkreuzkam-cap Oct 23, 2024
2c73d7d
Merge branch 'main' into EW-1047
mkreuzkam-cap Oct 24, 2024
8030d76
Move decryption to client factory.
mkreuzkam-cap Oct 24, 2024
c8baf32
Merge branch 'main' into EW-1047
mkreuzkam-cap Oct 25, 2024
f45c537
Change config of tsp client to differ from old sync.
mkreuzkam-cap Oct 25, 2024
441974d
add debug log.
mkreuzkam-cap Oct 28, 2024
585f48f
Merge branch 'main' into EW-1047
mkreuzkam-cap Oct 28, 2024
61df3cf
Add rabbitmq shutdown.
mkreuzkam-cap Oct 28, 2024
29a6336
more logging.
mkreuzkam-cap Oct 28, 2024
c8830e9
Add process.exit(0)
mkreuzkam-cap Oct 28, 2024
4f8d003
cleanup
mkreuzkam-cap Oct 28, 2024
0cf07b9
add console to sonar exclusion
mkreuzkam-cap Oct 28, 2024
b4a431c
Merge branch 'main' into EW-1047
mkreuzkam-cap Oct 28, 2024
4ca2e95
Add initial tsp sync job
mkreuzkam-cap Oct 29, 2024
ca2e5a1
Add config for sync to configmap.
mkreuzkam-cap Oct 30, 2024
4690f47
Add config map of api to tsp sync.
mkreuzkam-cap Oct 30, 2024
da31e71
Merge branch 'main' into EW-1047
mkreuzkam-cap Oct 30, 2024
e8c726d
add removal of tsp init job
mkreuzkam-cap Oct 30, 2024
c2a869d
Merge branch 'main' into EW-1047
mkreuzkam-cap Nov 4, 2024
41d8876
Merge branch 'main' into EW-1047
mkreuzkam-cap Nov 4, 2024
8f46b48
Apply suggestions from code review
mkreuzkam-cap Nov 5, 2024
5ea496e
add labels for init job
mkreuzkam-cap Nov 5, 2024
12c0461
Merge branch 'main' into EW-1047
mkreuzkam-cap Nov 5, 2024
6d4f32f
Job has to be removed.
mkreuzkam-cap Nov 5, 2024
0247b6e
Merge branch 'main' into EW-1047
mkreuzkam-cap Nov 6, 2024
5c3ba56
Split cronjob and init job configmap.
mkreuzkam-cap Nov 6, 2024
e4bba92
add " to var
mkreuzkam-cap Nov 6, 2024
b44fd67
use correct configmap in init job.
mkreuzkam-cap Nov 6, 2024
4b328fb
change order of configmaps
mkreuzkam-cap Nov 6, 2024
ec39bcb
increase data days to fetch.
mkreuzkam-cap Nov 6, 2024
7722083
Merge branch 'main' into EW-1047
mkreuzkam-cap Nov 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ data:
git clone https://github.com/hpi-schul-cloud/schulcloud-server.git
cd /schulcloud-server
git checkout {{ SCHULCLOUD_SERVER_IMAGE_TAG }}
npm install
npm install --include=dev
mamutmk5 marked this conversation as resolved.
Show resolved Hide resolved
until mongosh $DATABASE__URL --eval "print(\"waited for connection\")"
do
sleep 1
Expand Down Expand Up @@ -559,5 +559,43 @@ data:

# ========== End of the Instance seed data configuration section.

# ========== Start of TSP system creation
if [[ $SC_THEME == "thr" ]]; then
echo "Adding TSP system to systems collection"

TSP_SYSTEM_OAUTH_CLIENT_SECRET=$(node scripts/secret.js -s $AES_KEY -e $TSP_SYSTEM_OAUTH_CLIENT_SECRET)
mongosh $DATABASE__URL --quiet --eval 'db.systems.insertOne(
{
"_id": ObjectId("66d707f5c5202ba10c5e6256"),
"alias": "TSP",
"displayName": "Thüringer Schulportal",
"type": "oauth",
"provisioningStrategy": "tsp",
"oauthConfig": {
"clientId": "'$TSP_SYSTEM_OAUTH_CLIENT_ID'",
"clientSecret": "'$TSP_SYSTEM_OAUTH_CLIENT_SECRET'",
"tokenEndpoint": "https://test2.schulportal-thueringen.de/auth/realms/TIS/protocol/openid-connect/token",
"grantType": "authorization_code",
"scope": "openid",
"responseType": "code",
"redirectUri": "https://{{ NAMESPACE }}.thr.dbildungscloud.dev/api/v3/sso/oauth",
"authEndpoint": "https://test2.schulportal-thueringen.de/auth/realms/TIS/protocol/openid-connect/auth",
"provider": "tsp",
"jwksEndpoint": "https://test2.schulportal-thueringen.de/auth/realms/TIS/protocol/openid-connect/certs",
"issuer": "https://test2.schulportal-thueringen.de/auth/realms/TIS"
}
}
);'

echo "Successfully added TSP system"

echo "Running TSP Sync"
npm run nest:start:sync tsp

echo "Successfully ran TSP sync"
fi

# ========== End of TSP system creation

# Database indexes synchronization, it's crucial until we have all the entities in NestJS app.
npm run syncIndexes
1 change: 1 addition & 0 deletions apps/server/src/console/console.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ void bootstrap.init().then(async (app) => {
await app.init();
await bootstrap.boot();
await app.close();
process.exit(0);
mamutmk5 marked this conversation as resolved.
Show resolved Hide resolved
} catch (e) {
// eslint-disable-next-line no-console
console.error(e);
Expand Down
4 changes: 2 additions & 2 deletions apps/server/src/infra/tsp-client/tsp-client-config.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export interface TspClientConfig {
TSP_API_BASE_URL: string;
TSP_API_TOKEN_LIFETIME_MS: number;
TSP_API_CLIENT_BASE_URL: string;
TSP_API_CLIENT_TOKEN_LIFETIME_MS: number;
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ describe('TspClientFactory Integration', () => {
createMock<ConfigService>({
getOrThrow: (key: string) => {
switch (key) {
case 'TSP_API_BASE_URL':
case 'TSP_API_CLIENT_BASE_URL':
return 'https://test2.schulportal-thueringen.de/tip-ms/api';
case 'TSP_API_TOKEN_LIFETIME_MS':
case 'TSP_API_CLIENT_TOKEN_LIFETIME_MS':
return 30_000;
default:
throw new Error(`Unknown key: ${key}`);
Expand Down
12 changes: 10 additions & 2 deletions apps/server/src/infra/tsp-client/tsp-client-factory.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@ import { ServerConfig } from '@modules/server';
import { ConfigService } from '@nestjs/config';
import { Test, TestingModule } from '@nestjs/testing';
import axios from 'axios';
import { DefaultEncryptionService, EncryptionService } from '../encryption';
import { TspClientFactory } from './tsp-client-factory';

describe('TspClientFactory', () => {
let module: TestingModule;
let sut: TspClientFactory;
let configServiceMock: DeepMocked<ConfigService<ServerConfig, true>>;
let oauthAdapterServiceMock: DeepMocked<OauthAdapterService>;
let encryptionService: DeepMocked<EncryptionService>;

beforeAll(async () => {
module = await Test.createTestingModule({
Expand All @@ -26,22 +28,27 @@ describe('TspClientFactory', () => {
useValue: createMock<ConfigService<ServerConfig, true>>({
getOrThrow: (key: string) => {
switch (key) {
case 'TSP_API_BASE_URL':
case 'TSP_API_CLIENT_BASE_URL':
return faker.internet.url();
case 'TSP_API_TOKEN_LIFETIME_MS':
case 'TSP_API_CLIENT_TOKEN_LIFETIME_MS':
return faker.number.int();
default:
throw new Error(`Unknown key: ${key}`);
}
},
}),
},
{
provide: DefaultEncryptionService,
useValue: createMock<EncryptionService>(),
},
],
}).compile();

sut = module.get(TspClientFactory);
configServiceMock = module.get(ConfigService);
oauthAdapterServiceMock = module.get(OauthAdapterService);
encryptionService = module.get(DefaultEncryptionService);
});

afterAll(async () => {
Expand Down Expand Up @@ -118,6 +125,7 @@ describe('TspClientFactory', () => {

expect(response).toBeDefined();
expect(configServiceMock.getOrThrow).toHaveBeenCalledTimes(0);
expect(encryptionService.decrypt).toHaveBeenCalled();
});
});

Expand Down
13 changes: 8 additions & 5 deletions apps/server/src/infra/tsp-client/tsp-client-factory.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import { Injectable } from '@nestjs/common';
import { Inject, Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { OauthAdapterService } from '@src/modules/oauth';
import { OAuthGrantType } from '@src/modules/oauth/interface/oauth-grant-type.enum';
import { ClientCredentialsGrantTokenRequest } from '@src/modules/oauth/service/dto';
import * as jwt from 'jsonwebtoken';
import { DefaultEncryptionService, EncryptionService } from '../encryption';
import { Configuration, ExportApiFactory, ExportApiInterface } from './generated';
import { TspClientConfig } from './tsp-client-config';

Expand All @@ -25,10 +26,11 @@ export class TspClientFactory {

constructor(
private readonly oauthAdapterService: OauthAdapterService,
configService: ConfigService<TspClientConfig, true>
configService: ConfigService<TspClientConfig, true>,
@Inject(DefaultEncryptionService) private readonly encryptionService: EncryptionService
) {
this.baseUrl = configService.getOrThrow<string>('TSP_API_BASE_URL');
this.tokenLifetime = configService.getOrThrow<number>('TSP_API_TOKEN_LIFETIME_MS');
this.baseUrl = configService.getOrThrow<string>('TSP_API_CLIENT_BASE_URL');
this.tokenLifetime = configService.getOrThrow<number>('TSP_API_CLIENT_TOKEN_LIFETIME_MS');
}

public createExportClient(params: FactoryParams): ExportApiInterface {
Expand All @@ -51,9 +53,10 @@ export class TspClientFactory {
return this.cachedToken;
}

const clientSecret = this.encryptionService.decrypt(params.clientSecret);
const payload = new ClientCredentialsGrantTokenRequest({
client_id: params.clientId,
client_secret: params.clientSecret,
client_secret: clientSecret,
grant_type: OAuthGrantType.CLIENT_CREDENTIALS_GRANT,
});

Expand Down
3 changes: 2 additions & 1 deletion apps/server/src/infra/tsp-client/tsp-client.module.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import { OauthModule } from '@modules/oauth';
import { Module } from '@nestjs/common';
import { EncryptionModule } from '../encryption';
import { TspClientFactory } from './tsp-client-factory';

@Module({
imports: [OauthModule],
imports: [OauthModule, EncryptionModule],
providers: [TspClientFactory],
exports: [TspClientFactory],
})
Expand Down
4 changes: 2 additions & 2 deletions apps/server/src/modules/server/server.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -316,8 +316,8 @@ const config: ServerConfig = {
FEATURE_SANIS_GROUP_PROVISIONING_ENABLED: Configuration.get('FEATURE_SANIS_GROUP_PROVISIONING_ENABLED') as boolean,
FEATURE_AI_TUTOR_ENABLED: Configuration.get('FEATURE_AI_TUTOR_ENABLED') as boolean,
FEATURE_ROOMS_ENABLED: Configuration.get('FEATURE_ROOMS_ENABLED') as boolean,
TSP_API_BASE_URL: Configuration.get('TSP_API_BASE_URL') as string,
TSP_API_TOKEN_LIFETIME_MS: Configuration.get('TSP_API_TOKEN_LIFETIME_MS') as number,
TSP_API_CLIENT_BASE_URL: Configuration.get('TSP_API_CLIENT_BASE_URL') as string,
TSP_API_CLIENT_TOKEN_LIFETIME_MS: Configuration.get('TSP_API_CLIENT_TOKEN_LIFETIME_MS') as number,
TSP_SYNC_SCHOOL_LIMIT: Configuration.get('TSP_SYNC_SCHOOL_LIMIT') as number,
TSP_SYNC_SCHOOL_DAYS_TO_FETCH: Configuration.get('TSP_SYNC_SCHOOL_DAYS_TO_FETCH') as number,
TSP_SYNC_DATA_LIMIT: Configuration.get('TSP_SYNC_DATA_LIMIT') as number,
Expand Down
7 changes: 6 additions & 1 deletion config/default.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,12 @@
"default": "",
"description": "The key used to sign/verify TSP request tokens."
},
"TSP_API_TOKEN_LIFETIME_MS": {
"TSP_API_CLIENT_BASE_URL": {
mkreuzkam-cap marked this conversation as resolved.
Show resolved Hide resolved
"type": "string",
"default": "",
"description": "The TSP API base URL."
},
"TSP_API_CLIENT_TOKEN_LIFETIME_MS": {
"type": "number",
"default": "30000",
"description": "The TSP token lifetime in milliseconds."
Expand Down
2 changes: 1 addition & 1 deletion sonar-project.properties
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sonar.sources=.
sonar.tests=.
sonar.test.inclusions=**/*.spec.ts
sonar.exclusions=**/*.js,jest.config.ts,globalSetup.ts,globalTeardown.ts,**/*.app.ts,**/seed-data/*.ts,**/migrations/mikro-orm/*.ts,**/etherpad-api-client/**/*.ts,**/authorization-api-client/**/*.ts, **/course-api-client/**/*.ts,**/board-api-client/**/*.ts,**/generated/**/*.ts,**/room-api-client/**/*.ts,**/lessons-api-client/**/*.ts
sonar.coverage.exclusions=**/board-management.uc.ts,**/*.module.ts,**/*.factory.ts,**/migrations/mikro-orm/*.ts,**/globalSetup.ts,**/globalTeardown.ts,**/etherpad-api-client/**/*.ts,**/authorization-api-client/**/*.ts, **/course-api-client/**/*.ts,**/board-api-client/**/*.ts,**/generated/**/*.ts,**/room-api-client/**/*.ts
sonar.coverage.exclusions=**/board-management.uc.ts,**/*.module.ts,**/*.factory.ts,**/migrations/mikro-orm/*.ts,**/globalSetup.ts,**/globalTeardown.ts,**/etherpad-api-client/**/*.ts,**/authorization-api-client/**/*.ts, **/course-api-client/**/*.ts,**/board-api-client/**/*.ts,**/generated/**/*.ts,**/room-api-client/**/*.ts,apps/server/src/console/console.ts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why exlude apps/server/src/console/console.ts from the code coverrage ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There isn't any test for this file as far as I can see and inside it are several more exclusions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not write it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really scope of this ticket. Most of the console apps don't have tests

sonar.cpd.exclusions=**/controller/dto/**/*.ts,**/api/dto/**/*.ts,**/shared/testing/factory/*.factory.ts
sonar.javascript.lcov.reportPaths=merged-lcov.info
sonar.typescript.tsconfigPaths=tsconfig.json,src/apps/server/tsconfig.app.json
Loading