POC for automated testing of correct autorization on endpoints

[baszoetekouw]

I would like to have automatic tests to prevent bugs like #1457.
Ideally there would be a test that iterates over all roles in the platform, tries to access all endpoints with all roles and checks that only the correct roles have access.

This is a huge piece of work, so I'm unsure on how to take this on. We should probably start with something like role definitions to see which role there are in the first place.

Or we could approach it from a different direction, and introduce a framework that helps with the mapping from user/token to role to access rights, and migrate all API authorisation to use a more structural framework like this.

@oharsta do you have an idea of there is a somewhat doable way to achieve this?

[baszoetekouw]

For example would a framework like these work for us?

• https://github.com/casbin/pycasbin
• https://github.com/YosaiProject/yosai

[mrvanes]

Of python -> Open Policy Agent
https://opa-python.readthedocs.io/en/latest/

[oharsta]

@baszoetekouw perhaps it is a good idea to have a whiteboard session about this? What do we want to achieve, which problems it will solve and what would a MVP look like? There is an inherent risk of over-engineering this.

[logan-life]

@logan-life to plan a whiteboard session for this in October or November.

[logan-life]

Scheduling request sent for the whiteboard session.

[baszoetekouw]

Conclusion of whiteboarding session: authorization framework doesn't seem feasible and will be complex anyway (because the system itself if complex).
So we've decided to implement testing of all endpoints.

The tests should be automatically generated to iterate over all roles and all endpoints, and check if there is access.

Easy way to start: anonymous user, iterate over all endpoints, and test if we have access. Generate a seed specifically for these tests; define a yaml file that describes these desired results.

So, start a POC:

1. enumerate all endpoints
2. test all using an anonymous user
3. yaml to specify expected results
4. compare 2+3

After the poc, see what this brings us, and if we should expand this to iterate over other roles (OrgAPI user, logged in user, co admin, etc)

[logan-life]

Time-boxed effort for the POC = ~ 3-4 days

Potential benefits to realize:

• Security of endpoints (natuurlijk)
• Reviewing the JSON returns from endpoint tests may help us determine if/how we want to reduce the amount of data returned
• Identify possibilities for simplification / de-duplication

[oharsta]

Waiting for to be merged. https://github.com/SURFscz/SBS/tree/feature/automated-authz-testing-1461 is the branch with the automated end-points tests, and is created from the #1630 branch, which can not be merged until eduTeams fixes the refactored login flow.

[oharsta]

Can be tested. Is merged to main branch.

[mrvanes]

See https://github.com/SURFscz/SBS/tree/main/server/test/auth_endpoints

[mrvanes]

Ok