Development #3

narendra-sacumen · 2022-06-08T15:03:04Z

Description

Thank you for contributing to a JupiterOne integration!

Please include a summary of the change and which issue is fixed. Please also
include relevant motivation and context. List any dependencies that are required
for this change.

Summary

data mapping for meachine data

Relationship between account and machine is complete

vernability mapping according to the schema

userlogon api integration dependson machine Id

Changes for machine user mapping

Moved common params to env configuration file

Account to machine to device mapping changes. Hostagent mapping.

Completed machine to device/userEnpoint mapping

Added necessary comments to the newly written methods

Added all properties to the interface for future use

Type of change

Please leave any irrelevant options unchecked.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to
not work as expected)
This change requires a documentation update

Checklist

General Development Checklist:

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Integration Development Checklist:

Please leave any irrelevant options unchecked.

I have checked for additional permissions required to call any new API
endpoints, and have documented any additional permissions in
jupiterone.md, where necessary.
My changes properly paginate the target service provider's API
My changes properly handle rate limiting of the target service provider's
API
My new integration step is instrumented to execute in the correct order
using dependsOn
I have referred to the
JupiterOne data model
to ensure that any new entities/relationships, and relevant properties,
match the recommended model for this class of data
I have updated the CHANGELOG.md file to describe my changes
When changes include modifications to existing graph data ingestion, I've
reviewed all existing managed questions referencing the entities,
relationships, and their property names, to ensure those questions still
function with my changes.

Added the deleted files Data mapping from machine to device added few properties as per the discussion Changes done as per PR comments

…defender_dev into development

aiwilliams · 2022-06-22T01:19:03Z

src/ms-graph/__tests__/client.test.ts

+const config: IntegrationConfig = {
+  clientId: process.env.CLIENT_ID || 'a8626c1e-191d-4e8f-9cbc-a4a6e85104ac',
+  clientSecret:
+    process.env.CLIENT_SECRET || 'L3X8Q~BO57QkoAsXMSkevrmVR2qiNh.qKEuzucAt',


This appears to be a real secret and should not be included in source code.

It should be possible to use the test/config.ts with LOAD_ENV set to true, and your local .env configured with this secret value. Note how test/config.ts does this:

export const integrationConfig: IntegrationConfig = { clientId: process.env.CLIENT_ID || DEFAULT_CLIENT_ID, clientSecret: process.env.CLIENT_SECRET || DEFAULT_CLIENT_SECRET, tenant: process.env.TENANT || DEFAULT_TENANT, };

Here is similar code: https://github.com/JupiterOne/graph-microsoft-365/blob/a2cceb9734bf9715ccbc87a53c0a5e1fbc8dc96f/src/ms-graph/__tests__/client.test.ts#L6

When we need new recordings, we:

Delete existing recordings so that new recordings are made

Ensure .env has valid credentials

Run yarn test:env

In CI/CD, there should be no real request made thanks to the recordings, and therefore no need for the actual credentials.

aiwilliams · 2022-06-22T01:38:37Z

src/steps/ms-defender/clients/defenderClient.ts

+    if (process.env.FINDING_SEVERITY) {
+      filters.push(
+        '$filter=severity+eq+' +
+          process.env.FINDING_SEVERITY?.split(',')
+            .map((z) => "'" + z + "'")
+            .join('+or+severity+eq+'),
+      );
+    }
+    if (process.env.FINDIGS_LIMIT) {
+      filters.push('$top=' + process.env.FINDIGS_LIMIT);
+    }
+
+    if (filters && filters.length) {
+      filters = filters.join('&');
+    }
+
+    const url = `${process.env.BASE_URL_API}/machines/${input.machineId}/vulnerabilities?${filters}`;


All env vars should be defined on the src/config.ts instanceConfigFields, and read from the instance.config, not directly from process.env. The reason for this is that the managed runtime environment will allow users of JupiterOne to configure properties that are defined in the instanceConfigFields. The integration SDK will populate these values from the process.env automatically. See https://github.com/JupiterOne/sdk/blob/main/docs/integrations/development.md#instanceconfigfields.

aiwilliams · 2022-06-22T01:39:09Z

src/steps/ms-defender/clients/defenderClient.ts

+    if (process.env.FINDIGS_LIMIT) {
+      filters.push('$top=' + process.env.FINDIGS_LIMIT);
+    }


This has a typo and does not match the value in the .env.example file.

aiwilliams · 2022-06-22T03:08:09Z

src/steps/ms-defender/constants.ts

+import { entities as adEntities } from '../active-directory';
+
+export const steps: Record<string, string> = {
+  FETCH_MACHINE: 'Fetch machine',


These should be plural since they are processing a collection of resources.

Suggested change

FETCH_MACHINE: 'Fetch machine',

FETCH_MACHINES: 'Fetch machines',

aiwilliams · 2022-06-22T03:10:18Z

src/steps/ms-defender/constants.ts

+  },
+  DEVICE: {
+    resourceName: 'Device',
+    _type: 'user_endPoint',


All places that are currently endPoint should be endpoint (not camelCase).

aiwilliams · 2022-06-22T03:43:40Z

src/steps/ms-defender/user/index.ts

+>[] = [
+  {
+    id: steps.FETCH_USERS,
+    name: 'Ms defender machine logged in Users',


Suggested change

name: 'Ms defender machine logged in Users',

name: 'Fetch Machine Users',

aiwilliams · 2022-06-22T03:45:31Z

src/steps/ms-defender/user/__tests__/index.test.ts

+const config: IntegrationConfig = {
+  clientId: 'a8626c1e-191d-4e8f-9cbc-a4a6e85104ac',
+  clientSecret: 'L3X8Q~BO57QkoAsXMSkevrmVR2qiNh.qKEuzucAt',
+  tenant: '5a721b05-53ed-4ed9-be02-aed28f11edbd',
+};


More secret materials in git.

aiwilliams · 2022-06-22T03:46:55Z

src/steps/ms-defender/user/__tests__/__snapshots__/index.test.ts.snap

@@ -0,0 +1,3 @@
+// Jest Snapshot v1, https://goo.gl/fbAQLP
+
+exports[`fetchLogonUsers Should create an User entity correctly when log on User has the correct permissions: logOnUserEntitiesSuccessful 1`] = `Array []`;


Another empty snapshot, suggesting the test is doing nothing.

aiwilliams · 2022-06-22T03:48:26Z

src/steps/ms-defender/user/__tests__/index.test.ts

+    const logOnUserEntities = context.jobState.collectedEntities;
+    expect(logOnUserEntities.length).toBe(0);
+    expect(logOnUserEntities).toMatchGraphObjectSchema({
+      _class: entities.USER._class,
+    });
+    expect(logOnUserEntities).toMatchSnapshot('logOnUserEntitiesSuccessful');


With the combination of asserting there is nothing collected and an empty snapshot, I think this test is not working to prove the described outcome. I think this may be a similar situation with all the empty snapshots and related test code.

aiwilliams · 2022-06-22T03:49:13Z

src/steps/ms-defender/user/converters.ts

+export function createUserEntity(data: UserLogon): Entity {
+  return createIntegrationEntity({
+    entityData: {
+      source: {}, // removed due to size


Another case where I wonder if this was really a problem. 🤔

…he 2nd PR comments

…defender_dev into development

narendra-sacumen and others added 30 commits June 8, 2022 19:26

Folder strcuture changes

35d727c

Added the deleted files Data mapping from machine to device added few properties as per the discussion Changes done as per PR comments

Folder strcuture changes

538fec7

Added the deleted files Data mapping from machine to device added few properties as per the discussion Changes done as per PR comments

Folder strcuture changes

848cc66

Added the deleted files Data mapping from machine to device added few properties as per the discussion Changes done as per PR comments

Folder strcuture changes

bd0f2cb

Added the deleted files Data mapping from machine to device added few properties as per the discussion Changes done as per PR comments

Folder strcuture changes

02b250c

Added the deleted files Data mapping from machine to device added few properties as per the discussion Changes done as per PR comments

Ignoring node_modules from scan

cedd76b

Ignoring node_modules from scan

57259b5

Ignoring node_modules from scan

793174a

Ignoring node_modules from scan

2fa163f

Ignoring node_modules from scan

b54b126

Updated development.md file with necessary info

20ff4b7

Update filters config

54c96b2

Removed unnecessary file

55b52db

testcases for ms defender

b583179

removed package.lock.json

9cd5431

removed snapshort files

25e42b3

removed snapshort files

e4e6351

package.json

a9c5b0f

package.json

7a4383a

removed all unwanted files

3dfb191

changes in packege.json

61a0359

formated files

9f7bf0c

removed node modules

edcdf41

Merge branch 'development' of https://github.com/narendra-sacumen/ms_…

02a4d1d

…defender_dev into development

code changes in test cases

e665841

code changes in test cases

b01d48b

commited code in testcases

671ebb2

commited code in testcases

790c28f

changes in testcaases

e608d71

code changes

ea879e1

aiwilliams suggested changes Jun 22, 2022

View reviewed changes

devika-sacumen and others added 29 commits June 22, 2022 12:46

fixed typo errors & few configurations & unit testcase mentioned in t…

45cbae5

…he 2nd PR comments

few more changes in testcases comments fixed

50c6de9

BASE_API_URL is removed from env as it is required only in single file

0b8746c

BASE_API_URL is removed from env as it is required only in single file

8a7c14b

Removed env vars reading, added them to instanceconfig

d648ea2

Removed env vars reading, added them to instanceconfig

bbd2dec

snapshots fixes

24ba01b

snapshots fixes

72351b3

snapshots fixes

500bf6d

snapshots fixes

b56c11f

snapshots fixes

ae1e899

minor changes in snapshorts

99344c5

removed unwanted code

0f00f77

removed unwanted code

71b4892

removed unwanted code

c444e86

removed unwanted code

a813c21

removed unwanted code

2d07b0d

Documentation updates

1f9ad12

removed async from testcase

75fa1d2

removed async from testcase

b252a8e

testcase error resolved

b0b3d92

Merge branch 'development' of https://github.com/narendra-sacumen/ms_…

a2a133b

…defender_dev into development

added tests for inaccwssible and insufficient permission

ae36ac5

added tests for inaccwssible and insufficient permission

013372b

added tests for inaccwssible and insufficient permission

0d071d9

changes in testcase

8f94c2d

removed commented code

c2b74a5

snapshots for all testcases

bdddeec

changed testases relationships

a5f1884

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Development #3

Development #3

narendra-sacumen commented Jun 8, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

aiwilliams Jun 22, 2022

	FETCH_MACHINE: 'Fetch machine',
	FETCH_MACHINES: 'Fetch machines',

	name: 'Ms defender machine logged in Users',
	name: 'Fetch Machine Users',

		@@ -0,0 +1,3 @@
		// Jest Snapshot v1, https://goo.gl/fbAQLP

		exports[`fetchLogonUsers Should create an User entity correctly when log on User has the correct permissions: logOnUserEntitiesSuccessful 1`] = `Array []`;

Development #3

Are you sure you want to change the base?

Development #3

Conversation

narendra-sacumen commented Jun 8, 2022

Description

Summary

Type of change

Checklist

General Development Checklist:

Integration Development Checklist:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment