Do not be case-sensitive when checking http headers #2277

williamlardier · 2024-12-23T10:06:20Z

This is affecting policies evaluation in both the S3 and IAM services, when there is a condition on the IP.
No impact for S3C because the header is already properly formatted, but not in Zenko.

Issue: ARSN-453

bert-e · 2024-12-23T10:06:24Z

Hello williamlardier,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2024-12-23T10:09:15Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

Issue: ARSN-453

francoisferrand

do we really need the extra conversion to lowercase, can't we just fix the 'X-Forwarded-For' where it is defined instead?

This adds a (small) overhead to every check, slightly increasing latency... and small things add up, over time...

francoisferrand · 2024-12-26T11:52:50Z

lib/policyEvaluator/requestUtils.ts

+            // Request headers in nodejs are lower-cased, so we should not
+            // be case-sentive when looking for the header, as http headers
+            // are case-insensitive.
+            const ipFromHeader = request.headers[extractClientIPFromHeader.toLowerCase()]?.toString();


looking a few lines above, we already retrieve the x-forwarded-for header:

const clientIp = (requestConfig ? remoteAddress : request.headers['x-forwarded-for'] || remoteAddress)?.toString() ?? '';

this seems inconsistent, please take the chance to review and clean this...
Esp. if requestConfig is not provided, we kind of bindly trust the x-forwarded-for header : is this really expected?

do we really need the extra conversion to lowercase, can't we just fix the 'X-Forwarded-For' where it is defined instead?

We could do it but the headers are case-insensitive anyway, so the current code is not following the standard and we might have more issues in the future...

looking a few lines above, we already retrieve the x-forwarded-for header:
const clientIp = (requestConfig ? remoteAddress : request.headers['x-forwarded-for'] || remoteAddress)?.toString() ?? '';
this seems inconsistent, please take the chance to review and clean this...
Esp. if requestConfig is not provided, we kind of bindly trust the x-forwarded-for header : is this really expected?

I am not aware of any deployment today that does not set the requestConfig, but this header retrieval was introduced in https://scality.atlassian.net/browse/ARSN-57, so here: https://github.com/scality/Arsenal/pull/1693/files#diff-ca8ab692737b69e1ab977a7a6104be671972410cafadea45b119e8259070ad1bR11

If the question is whether we should trust the header blindly or not: as long as the proxy is not trusted we should not trust the header. I will remove this logic to be on the safe side.

Previously we were not using IP conditions so it was only about displaying the info in logs, hence the previous change.

extractClientIPFromHeader probably does not work if not equal to x-forwarded-for

Please elaborate what you mean by "does not work": it's a string field, that can be any header: e.g., we can have a custom header set by nginx and use it.

extractClientIPFromHeader probably does not work if not equal to x-forwarded-for

Please elaborate what you mean by "does not work": it's a string field, that can be any header: e.g., we can have a custom header set by nginx and use it.

"does not work" was too quick, I missed the condition on requestConfig

extractClientIPFromHeader probably does not work if not equal to x-forwarded-for

Issue: ARSN-453

williamlardier · 2024-12-26T15:02:52Z

/create_integration_branches

bert-e · 2024-12-26T15:05:40Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge bugfix/ARSN-453-fix-ipchecks into
development/7.10
w/7.70/bugfix/ARSN-453-fix-ipchecks will be merged into development/7.70
w/8.1/bugfix/ARSN-453-fix-ipchecks will be merged into development/8.1
w/8.2/bugfix/ARSN-453-fix-ipchecks will be merged into development/8.2

The following branches will NOT be impacted:

development/6.4
development/7.4

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: create_integration_branches

bert-e · 2024-12-26T15:05:41Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

The following options are set: create_integration_branches

williamlardier · 2024-12-27T08:28:11Z

/approve

bert-e · 2024-12-27T08:28:25Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/7.10
✔️ development/7.70
✔️ development/8.1
✔️ development/8.2

The following branches have NOT changed:

development/6.4
development/7.4

Please check the status of the associated issue ARSN-453.

Goodbye williamlardier.

The following options are set: approve, create_integration_branches

williamlardier force-pushed the bugfix/ARSN-453-fix-ipchecks branch 3 times, most recently from ffd13b2 to eb725d6 Compare December 23, 2024 10:08

scality deleted a comment from bert-e Dec 23, 2024

Do not be case-sensitive when checking http headers

89a7f0e

Issue: ARSN-453

williamlardier force-pushed the bugfix/ARSN-453-fix-ipchecks branch from eb725d6 to 89a7f0e Compare December 23, 2024 10:10

KillianG approved these changes Dec 23, 2024

View reviewed changes

francoisferrand previously approved these changes Dec 26, 2024

View reviewed changes

francoisferrand reviewed Dec 26, 2024

View reviewed changes

Do not trust the x forwarded for header if not from a trusted proxy

5aadd36

Issue: ARSN-453

williamlardier requested a review from francoisferrand December 26, 2024 13:00

Remove old TODO on getClientIp and remove duplicate logic

753fea6

Issue: ARSN-453

francoisferrand approved these changes Dec 26, 2024

View reviewed changes

Bump Arsenal version

ad12532

Issue: ARSN-453

scality deleted a comment from bert-e Dec 26, 2024

bert-e merged commit ad12532 into development/7.10 Dec 27, 2024
5 checks passed

bert-e deleted the bugfix/ARSN-453-fix-ipchecks branch December 27, 2024 08:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not be case-sensitive when checking http headers #2277

Do not be case-sensitive when checking http headers #2277

williamlardier commented Dec 23, 2024

bert-e commented Dec 23, 2024

bert-e commented Dec 23, 2024

francoisferrand left a comment

francoisferrand Dec 26, 2024 •

edited

Loading

williamlardier Dec 26, 2024

williamlardier Dec 26, 2024 •

edited

Loading

williamlardier Dec 26, 2024

francoisferrand Dec 26, 2024

williamlardier commented Dec 26, 2024

bert-e commented Dec 26, 2024

bert-e commented Dec 26, 2024

williamlardier commented Dec 27, 2024

bert-e commented Dec 27, 2024

Do not be case-sensitive when checking http headers #2277

Do not be case-sensitive when checking http headers #2277

Conversation

williamlardier commented Dec 23, 2024

bert-e commented Dec 23, 2024

Hello williamlardier,

bert-e commented Dec 23, 2024

Request integration branches

francoisferrand left a comment

Choose a reason for hiding this comment

francoisferrand Dec 26, 2024 • edited Loading

Choose a reason for hiding this comment

williamlardier Dec 26, 2024

Choose a reason for hiding this comment

williamlardier Dec 26, 2024 • edited Loading

Choose a reason for hiding this comment

williamlardier Dec 26, 2024

Choose a reason for hiding this comment

francoisferrand Dec 26, 2024

Choose a reason for hiding this comment

williamlardier commented Dec 26, 2024

bert-e commented Dec 26, 2024

Integration data created

bert-e commented Dec 26, 2024

Waiting for approval

williamlardier commented Dec 27, 2024

bert-e commented Dec 27, 2024

francoisferrand Dec 26, 2024 •

edited

Loading

williamlardier Dec 26, 2024 •

edited

Loading