Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(feat) [NAN-677] enforce cap when calling nangoauth #1950

Merged
merged 15 commits into from
Apr 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions packages/server/lib/controllers/connection.controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import {
environmentService,
accountService,
connectionCreated as connectionCreatedHook,
connectionCreationStartCapCheck as connectionCreationStartCapCheckHook,
slackNotificationService
} from '@nangohq/shared';
import { getUserAccountAndEnvironmentFromSession } from '../utils/utils.js';
Expand Down Expand Up @@ -527,6 +528,21 @@ class ConnectionController {
return;
}

const account = await accountService.getAccountById(accountId);

if (!account) {
errorManager.errRes(res, 'unknown_account');
return;
}

if (account.is_capped && provider_config_key) {
const isCapped = await connectionCreationStartCapCheckHook({ providerConfigKey: provider_config_key, environmentId });
if (isCapped) {
errorManager.errRes(res, 'resource_capped');
return;
}
}
bodinsamuel marked this conversation as resolved.
Show resolved Hide resolved

const template = await configService.getTemplate(provider);

let oAuthCredentials: ImportedCredentials;
Expand Down Expand Up @@ -732,6 +748,13 @@ class ConnectionController {
accountId
);

if (imported) {
updatedConnection = imported;
runHook = true;
}
} else if (template.auth_mode === ProviderAuthModes.None) {
const [imported] = await connectionService.upsertUnauthConnection(connection_id, provider_config_key, provider, environmentId, accountId);

if (imported) {
updatedConnection = imported;
runHook = true;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
telemetry,
MetricTypes
} from '@nangohq/shared';
import { NANGO_ADMIN_UUID } from './account.controller.js';
import { NANGO_ADMIN_UUID } from '../controllers/account.controller.js';
import tracer from 'dd-trace';

const logger = getLogger('AccessMiddleware');
Expand Down
37 changes: 37 additions & 0 deletions packages/server/lib/middleware/resource-capping.middleware.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import type { Request, Response, NextFunction } from 'express';
import {
accountService,
getEnvironmentAndAccountId,
errorManager,
connectionCreationStartCapCheck as connectionCreationStartCapCheckHook
} from '@nangohq/shared';

export const authCheck = async (req: Request, res: Response, next: NextFunction) => {
const { success, error, response } = await getEnvironmentAndAccountId(res, req);

if (!success || response === null) {
errorManager.errResFromNangoErr(res, error);
return;
}

const { accountId, environmentId } = response;

const account = await accountService.getAccountById(accountId);
Copy link
Collaborator

@TBonnin TBonnin Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are making more queries to the db on each request by adding a middleware which is doing something similar to the access.middleware, ie: fetching the account and environment from the db. Can we fetch the account.is_capped as part of the access.middleware and save it into the res.locals like we do with accountId and environmentId so it can be access down the line by controllers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea in theory, but the issue is that the value of is_capped can change via the api and then the value stored in the res.locals is stale and potentially inaccurate. Since the user changing the is_capped setting is different than the user who has the capping nor not.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

res.locals is only stored for the current request if that's your worry

Copy link
Member Author

@khaliqgant khaliqgant Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, but still this is making it more complicated than it needs since right now the only controller it goes through is the connection.controller when it is imported. If it went into the access.middleware instead of the capping.middleware then the logic to check the cap would need to go through:

  • oauth.controller
  • unauth.controller
  • apiAuth.controller
  • appStoreController


if (!account) {
errorManager.errRes(res, 'unknown_account');
return;
}

const { providerConfigKey } = req.params;

if (account.is_capped && providerConfigKey) {
const isCapped = await connectionCreationStartCapCheckHook({ providerConfigKey, environmentId });
if (isCapped) {
errorManager.errRes(res, 'resource_capped');
return;
}
}

next();
};
bodinsamuel marked this conversation as resolved.
Show resolved Hide resolved
10 changes: 6 additions & 4 deletions packages/server/lib/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import connectionController from './controllers/connection.controller.js';
import authController from './controllers/auth.controller.js';
import unAuthController from './controllers/unauth.controller.js';
import appStoreAuthController from './controllers/appStoreAuth.controller.js';
import authMiddleware from './controllers/access.middleware.js';
import authMiddleware from './middleware/access.middleware.js';
import userController from './controllers/user.controller.js';
import proxyController from './controllers/proxy.controller.js';
import activityController from './controllers/activity.controller.js';
Expand All @@ -19,7 +19,8 @@ import apiAuthController from './controllers/apiAuth.controller.js';
import appAuthController from './controllers/appAuth.controller.js';
import onboardingController from './controllers/onboarding.controller.js';
import webhookController from './controllers/webhook.controller.js';
import { rateLimiterMiddleware } from './controllers/ratelimit.middleware.js';
import { rateLimiterMiddleware } from './middleware/ratelimit.middleware.js';
import { authCheck } from './middleware/resource-capping.middleware.js';
import path from 'path';
import { dirname } from './utils/utils.js';
import type { WebSocket } from 'ws';
Expand Down Expand Up @@ -50,7 +51,7 @@ setupAuth(app);

const apiAuth = [authMiddleware.secretKeyAuth.bind(authMiddleware), rateLimiterMiddleware];
const adminAuth = [authMiddleware.secretKeyAuth.bind(authMiddleware), authMiddleware.adminKeyAuth.bind(authMiddleware), rateLimiterMiddleware];
const apiPublicAuth = [authMiddleware.publicKeyAuth.bind(authMiddleware), rateLimiterMiddleware];
const apiPublicAuth = [authMiddleware.publicKeyAuth.bind(authMiddleware), authCheck, rateLimiterMiddleware];
const webAuth =
isCloud || isEnterprise
? [passport.authenticate('session'), authMiddleware.sessionAuth.bind(authMiddleware), rateLimiterMiddleware]
Expand Down Expand Up @@ -89,11 +90,12 @@ await oAuthSessionService.clearStaleSessions();
app.get('/health', (_, res) => {
res.status(200).send({ result: 'ok' });
});

app.route('/oauth/callback').get(oauthController.oauthCallback.bind(oauthController));
app.route('/webhook/:environmentUuid/:providerConfigKey').post(webhookController.receive.bind(proxyController));
app.route('/app-auth/connect').get(appAuthController.connect.bind(appAuthController));
app.route('/oauth/connect/:providerConfigKey').get(apiPublicAuth, oauthController.oauthRequest.bind(oauthController));
app.route('/oauth2/auth/:providerConfigKey').post(apiPublicAuth, oauthController.oauth2RequestCC.bind(oauthController));
app.route('/webhook/:environmentUuid/:providerConfigKey').post(webhookController.receive.bind(proxyController));
app.route('/api-auth/api-key/:providerConfigKey').post(apiPublicAuth, apiAuthController.apiKey.bind(apiAuthController));
app.route('/api-auth/basic/:providerConfigKey').post(apiPublicAuth, apiAuthController.basic.bind(apiAuthController));
app.route('/app-store-auth/:providerConfigKey').post(apiPublicAuth, appStoreAuthController.auth.bind(appStoreAuthController));
Expand Down
36 changes: 36 additions & 0 deletions packages/shared/lib/hooks/hooks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,45 @@ import integrationPostConnectionScript from '../integrations/scripts/connection/
import webhookService from '../services/notification/webhook.service.js';
import { SpanTypes } from '../utils/telemetry.js';
import { isCloud, isLocal, isEnterprise } from '../utils/temp/environment/detection.js';
import { getSyncConfigsWithConnections } from '../services/sync/config/config.service.js';
import type { Result } from '../utils/result.js';
import { resultOk, resultErr } from '../utils/result.js';
import { NangoError } from '../utils/error.js';
import { getLogger } from '../utils/temp/logger.js';

const logger = getLogger('hooks');

const CONNECTIONS_WITH_SCRIPTS_CAP_LIMIT = 3;

export const connectionCreationStartCapCheck = async ({
Copy link
Collaborator

@TBonnin TBonnin Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need a new function? Can we not call syncService.getSyncsByProviderConfigKey and count the connections?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That method would be good, but we need both to check against syncs and actions.

providerConfigKey,
environmentId
}: {
providerConfigKey: string | undefined;
environmentId: number;
}): Promise<boolean> => {
if (!providerConfigKey) {
return false;
}

const scriptConfigs = await getSyncConfigsWithConnections(providerConfigKey, environmentId);

const reachedCap = false;

if (scriptConfigs.length > 0) {
for (const script of scriptConfigs) {
const { connections } = script;

if (connections.length >= CONNECTIONS_WITH_SCRIPTS_CAP_LIMIT) {
//reachedCap = true;
logger.info(`Reached cap for providerConfigKey: ${providerConfigKey} and environmentId: ${environmentId}`);
break;
}
}
}

return reachedCap;
};

export const connectionCreated = async (
connection: RecentlyCreatedConnection,
Expand Down
38 changes: 38 additions & 0 deletions packages/shared/lib/services/sync/config/config.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,44 @@ export async function getSyncConfigsWithConnectionsByEnvironmentId(environment_i
return result;
}

export async function getSyncConfigsWithConnections(
providerConfigKey: string,
environment_id: number
): Promise<{ connections: { connection_id: string }[]; provider: string; unique_key: string }[]> {
const result = await db.knex
.select(
`${TABLE}.id`,
'_nango_configs.provider',
'_nango_configs.unique_key',
db.knex.raw(
`(
SELECT json_agg(
json_build_object(
'connection_id', _nango_connections.connection_id
)
)
FROM _nango_connections
WHERE _nango_configs.environment_id = _nango_connections.environment_id
AND _nango_configs.unique_key = _nango_connections.provider_config_key
AND _nango_configs.deleted = false
AND _nango_connections.deleted = false
) as connections
`
)
)
.from<SyncConfig>(TABLE)
.join('_nango_configs', `${TABLE}.nango_config_id`, '_nango_configs.id')
.where({
'_nango_configs.environment_id': environment_id,
'_nango_configs.unique_key': providerConfigKey,
active: true,
'_nango_configs.deleted': false,
[`${TABLE}.deleted`]: false
});

return result;
}

/**
* Get Sync Configs By Provider Key
* @desc grab all the sync configs by a provider key
Expand Down
7 changes: 7 additions & 0 deletions packages/shared/lib/utils/error.ts
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,13 @@ export class NangoError extends Error {
this.message = `Missing an account name for account login/signup.`;
break;

case 'resource_capped':
this.status = 400;
// TODO docs link
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending from Bastien.

this.message =
'You have reached the maximum number of integrations with active scripts. Upgrade or deactivate the scripts to create more connections (https://docs.nango.dev/relevant-link).';
break;

default:
this.status = 500;
this.type = 'unhandled_' + type;
Expand Down
Loading